This model is a fine-tuned version of Qwen/Qwen3-14B on the selected_svamp_v2_dpo_train dataset. It achieves the following results on the evaluation set:

Loss: 0.0109
Rewards/chosen: -0.5415
Rewards/rejected: -8.0294
Rewards/accuracies: 1.0
Rewards/margins: 7.4879
Logps/chosen: -173.3236
Logps/rejected: -248.0356
Logits/chosen: -1.6494
Logits/rejected: -1.4102

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Logits/chosen	Logits/rejected	Logps/chosen	Logps/rejected	Validation Loss	Rewards/accuracies	Rewards/chosen	Rewards/margins	Rewards/rejected
0.6075	0.0846	50	-1.6004	-1.2282	-166.2650	-169.2240	0.5693	0.8355	0.1644	0.3126	-0.1483
0.2627	0.1692	100	-1.5548	-1.1985	-160.3058	-178.4771	0.2854	0.9583	0.7603	1.8339	-1.0736
0.1038	0.2538	150	-1.5232	-1.1920	-160.3985	-194.5819	0.1231	0.9956	0.7510	3.4351	-2.6840
0.0933	0.3384	200	-1.5324	-1.2238	-161.8196	-207.4299	0.0717	1.0	0.6089	4.5778	-3.9688
0.0241	0.4230	250	-1.5792	-1.2965	-165.2303	-220.0915	0.0398	1.0	0.2679	5.5029	-5.2350
0.0214	0.5076	300	-1.5998	-1.3376	-168.5743	-231.1305	0.0241	1.0	-0.0665	6.2724	-6.3389
0.0127	0.5922	350	-1.6398	-1.3879	-171.4349	-239.4612	0.0178	1.0	-0.3526	6.8194	-7.1720
0.0281	0.6768	400	-1.6383	-1.3906	-171.7702	-242.7881	0.0140	1.0	-0.3861	7.1185	-7.5047
0.0242	0.7614	450	-1.6487	-1.4057	-172.5759	-245.9139	0.0117	1.0	-0.4667	7.3505	-7.8172
0.0021	0.8460	500	-1.6504	-1.4106	-173.3175	-247.6053	0.0112	1.0	-0.5409	7.4455	-7.9864
0.0086	0.9306	550	-1.6523	-1.4131	-173.3299	-247.9987	0.0110	1.0	-0.5421	7.4836	-8.0257

Framework versions

PEFT 0.18.1
Transformers 4.56.2
Pytorch 2.7.1+cu128
Datasets 3.6.0
Tokenizers 0.22.2

kyleliu789/qwen_3_14b_svampv2_full_no_md

qwen_3_14b_svampv2_full_no_md

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

kyleliu789/
qwen_3_14b_svampv2_full_no_md