This model is a fine-tuned version of Qwen/Qwen3-14B on the selected_svamp_v2_dpo_train dataset. It achieves the following results on the evaluation set:

Loss: 0.0001
Rewards/chosen: 4.4377
Rewards/rejected: -6.7316
Rewards/accuracies: 1.0
Rewards/margins: 11.1693
Logps/chosen: -131.1872
Logps/rejected: -206.7433
Logits/chosen: -2.2839
Logits/rejected: -1.3981

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 8
eval_batch_size: 4
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/chosen	Logps/rejected	Logits/chosen	Logits/rejected
0.647	0.2525	25	0.5979	0.2372	0.0344	0.9737	0.2028	-173.1918	-139.0830	-2.3107	-1.2143
0.0651	0.5051	50	0.0361	2.9663	-0.6560	1.0	3.6224	-145.9008	-145.9877	-2.2702	-1.2320
0.0009	0.7576	75	0.0007	4.2662	-4.2486	1.0	8.5148	-132.9020	-181.9128	-2.2788	-1.3570
0.0005	1.0101	100	0.0003	4.3941	-5.4902	1.0	9.8843	-131.6231	-194.3293	-2.2822	-1.3816
0.0002	1.2626	125	0.0002	4.4219	-5.9145	1.0	10.3364	-131.3451	-198.5719	-2.2792	-1.3851
0.0001	1.5152	150	0.0002	4.4277	-6.2349	1.0	10.6626	-131.2871	-201.7766	-2.2852	-1.3915
0.0001	1.7677	175	0.0001	4.4220	-6.4274	1.0	10.8494	-131.3439	-203.7010	-2.2821	-1.3941
0.0001	2.0202	200	0.0001	4.4225	-6.5684	1.0	10.9909	-131.3389	-205.1111	-2.2863	-1.3988
0.0002	2.2727	225	0.0001	4.4185	-6.6624	1.0	11.0810	-131.3790	-206.0516	-2.2835	-1.3964
0.0001	2.5253	250	0.0001	4.4289	-6.7039	1.0	11.1328	-131.2749	-206.4658	-2.2834	-1.3977
0.0001	2.7778	275	0.0001	4.4204	-6.7445	1.0	11.1650	-131.3599	-206.8725	-2.2860	-1.3982

Framework versions

PEFT 0.18.1
Transformers 4.56.2
Pytorch 2.7.1+cu128
Datasets 3.6.0
Tokenizers 0.22.2

kyleliu789/qwen3-14b-svamp-dpo-qlora-all-caps-1e-5

qwen3_14b_dpo_qlora_svamp_v2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

kyleliu789/
qwen3-14b-svamp-dpo-qlora-all-caps-1e-5