This model is a fine-tuned version of Qwen/Qwen3-14B on the selected_svamp_v2_dpo_train dataset. It achieves the following results on the evaluation set:

Loss: 0.0007
Rewards/chosen: 3.7231
Rewards/rejected: -4.7614
Rewards/accuracies: 1.0
Rewards/margins: 8.4844
Logps/chosen: -138.3334
Logps/rejected: -187.0409
Logits/chosen: -2.2954
Logits/rejected: -1.3468

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 8
eval_batch_size: 4
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/chosen	Logps/rejected	Logits/chosen	Logits/rejected
0.6947	0.2525	25	0.6612	0.0601	-0.0064	0.8158	0.0665	-174.9632	-139.4917	-2.3103	-1.2110
0.434	0.5051	50	0.3834	0.9455	0.1693	1.0	0.7761	-166.1096	-137.7340	-2.3035	-1.2157
0.09	0.7576	75	0.0716	2.4903	-0.3306	1.0	2.8208	-150.6615	-142.7328	-2.2870	-1.2322
0.0078	1.0101	100	0.0055	3.4526	-2.2277	1.0	5.6803	-141.0379	-161.7039	-2.2856	-1.2868
0.0022	1.2626	125	0.0023	3.6224	-3.1778	1.0	6.8002	-139.3398	-171.2052	-2.2931	-1.3198
0.001	1.5152	150	0.0015	3.6756	-3.7633	1.0	7.4389	-138.8080	-177.0600	-2.2950	-1.3332
0.001	1.7677	175	0.0011	3.6994	-4.1324	1.0	7.8318	-138.5699	-180.7509	-2.2987	-1.3410
0.0008	2.0202	200	0.0009	3.7094	-4.4536	1.0	8.1630	-138.4702	-183.9631	-2.2958	-1.3439
0.001	2.2727	225	0.0008	3.7238	-4.6157	1.0	8.3396	-138.3260	-185.5847	-2.2964	-1.3490
0.0009	2.5253	250	0.0007	3.7310	-4.7337	1.0	8.4647	-138.2543	-186.7645	-2.2958	-1.3485
0.0008	2.7778	275	0.0007	3.7291	-4.7426	1.0	8.4717	-138.2728	-186.8530	-2.2992	-1.3509

Framework versions

PEFT 0.18.1
Transformers 4.56.2
Pytorch 2.7.1+cu128
Datasets 3.6.0
Tokenizers 0.22.2

kyleliu789/qwen3-14b-svamp-dpo-qlora-all-caps

qwen3_14b_dpo_qlora_svamp_v2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

kyleliu789/
qwen3-14b-svamp-dpo-qlora-all-caps