This model is a fine-tuned version of akshay-sked/qwen3-14b-svamp-sft on the svamp_final_dpo_train dataset. It achieves the following results on the evaluation set:

Loss: 0.2312
Rewards/chosen: 1.4280
Rewards/rejected: -2.2705
Rewards/accuracies: 0.875
Rewards/margins: 3.6986
Logps/chosen: -229.8555
Logps/rejected: -244.6095
Logits/chosen: -1.0407
Logits/rejected: -0.6491

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 16
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/chosen	Logps/rejected	Logits/chosen	Logits/rejected
0.6205	0.2538	50	0.5905	0.0116	-0.2220	0.8421	0.2336	-244.0198	-224.1245	-1.0900	-0.6862
0.3823	0.5076	100	0.3367	0.2467	-1.6893	0.8717	1.9360	-241.6687	-238.7974	-1.0628	-0.6642
0.3276	0.7614	150	0.2943	0.5368	-2.3288	0.8651	2.8656	-238.7676	-245.1919	-1.0503	-0.6563
0.2355	1.0152	200	0.2611	1.0175	-2.0767	0.8684	3.0942	-233.9607	-242.6709	-1.0428	-0.6520
0.1452	1.2690	250	0.2588	1.3511	-2.0115	0.8717	3.3626	-230.625	-242.0193	-1.0398	-0.6503
0.2727	1.5228	300	0.2473	1.5908	-1.2269	0.8651	2.8177	-228.2282	-234.1732	-1.0386	-0.6496
0.2379	1.7766	350	0.2461	1.3454	-2.2200	0.8618	3.5654	-230.6818	-244.1037	-1.0399	-0.6488
0.1555	2.0305	400	0.2404	1.2508	-2.5865	0.8882	3.8372	-231.6284	-247.7691	-1.0412	-0.6501
0.2044	2.2843	450	0.2341	1.4322	-2.2009	0.8849	3.6331	-229.8141	-243.9135	-1.0404	-0.6484
0.1274	2.5381	500	0.2314	1.4235	-2.2426	0.8783	3.6661	-229.9008	-244.3304	-1.0391	-0.6482
0.2238	2.7919	550	0.2319	1.4203	-2.2936	0.8849	3.7139	-229.9328	-244.8397	-1.0393	-0.6478

Framework versions

PEFT 0.18.1
Transformers 4.56.2
Pytorch 2.7.1+cu128
Datasets 3.6.0
Tokenizers 0.22.2

akshay-sked/qwen3-14b-sft-full-qlora-dpo

qwen3_14b_svamp_sft_then_combined_dpo

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

akshay-sked/
qwen3-14b-sft-full-qlora-dpo