This model is a fine-tuned version of Qwen/Qwen3-14B on the selected_svamp_v2_dpo_train dataset. It achieves the following results on the evaluation set:

Loss: 0.5029
Rewards/chosen: -0.0454
Rewards/rejected: -0.0652
Rewards/accuracies: 1.0
Rewards/margins: 0.0198
Logps/chosen: -0.4543
Logps/rejected: -0.6519
Logits/chosen: -1.8621
Logits/rejected: -0.7316
Sft Loss: 0.4543
Odds Ratio Loss: 0.4861

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/chosen	Logps/rejected	Logits/chosen	Logits/rejected	Sft Loss	Odds Ratio Loss
0.9571	0.5076	50	0.9569	-0.0891	-0.0937	0.6711	0.0046	-0.8911	-0.9372	-2.1950	-1.0490	0.8911	0.6578
0.6783	1.0102	100	0.6756	-0.0619	-0.0751	0.9474	0.0132	-0.6187	-0.7508	-1.9782	-0.8322	0.6187	0.5693
0.4973	1.5178	150	0.5698	-0.0517	-0.0685	0.9868	0.0168	-0.5173	-0.6852	-1.8840	-0.7568	0.5173	0.5245
0.471	2.0203	200	0.5235	-0.0474	-0.0662	0.9868	0.0188	-0.4736	-0.6617	-1.8655	-0.7373	0.4736	0.4986
0.4595	2.5279	250	0.5052	-0.0457	-0.0654	1.0	0.0197	-0.4565	-0.6537	-1.8616	-0.7334	0.4565	0.4870

Framework versions

PEFT 0.18.1
Transformers 4.56.2
Pytorch 2.7.1+cu128
Datasets 3.6.0
Tokenizers 0.22.2

kyleliu789/qwen3-14B-svamp-all-caps-orpo

qwen3_14b_orpo_qlora_svamp_v2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

kyleliu789/
qwen3-14B-svamp-all-caps-orpo