This model is a fine-tuned version of hZzy/qwen2.5-0.5b-sft-news-IFT on the hZzy/train_pairwise_weighted dataset. It achieves the following results on the evaluation set:

Loss: 0.6921
Logps: -91.0309
Logits: -2.1203
Objective: 0.6893
Dpo Loss: 0.6893
Regularize: 0.6893
Ranking Simple: 0.5559
Ranking Idealized: 0.6030
Ranking Idealized Expo: 0.5223
Wo Beta: 7.4262

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 144
total_eval_batch_size: 12
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Logps	Logits	Objective	Dpo Loss	Regularize	Ranking Simple	Ranking Idealized	Ranking Idealized Expo	Wo Beta
0.6785	0.1417	50	0.6813	-90.8716	-1.6022	0.6843	0.6843	0.6843	0.5259	0.6030	0.5223	7.8749
0.618	0.2834	100	0.6733	-98.8899	-1.7799	0.6766	0.6766	0.6766	0.5399	0.6030	0.5223	7.7840
0.5667	0.4251	150	0.6866	-99.1230	-1.8072	0.6829	0.6829	0.6829	0.5409	0.6030	0.5223	7.8533
0.5214	0.5668	200	0.6901	-99.5388	-1.8894	0.6904	0.6904	0.6904	0.5445	0.6030	0.5223	7.6995
0.4922	0.7085	250	0.6976	-82.7973	-1.9880	0.6916	0.6916	0.6916	0.5476	0.6030	0.5223	7.8790
0.4535	0.8503	300	0.6921	-91.0309	-2.1203	0.6893	0.6893	0.6893	0.5559	0.6030	0.5223	7.4262
0.423	0.9920	350	0.7057	-88.1615	-1.9880	0.6959	0.6959	0.6959	0.5549	0.6030	0.5223	7.9979
0.2847	1.1337	400	0.7315	-101.6926	-2.0862	0.7281	0.7281	0.7281	0.5424	0.6030	0.5223	8.6326
0.2991	1.2754	450	0.7008	-92.7942	-1.8470	0.6980	0.6980	0.6980	0.5621	0.6030	0.5223	8.2584
0.3065	1.4171	500	0.7180	-96.6747	-2.0065	0.7147	0.7147	0.7147	0.5554	0.6030	0.5223	8.2522
0.2895	1.5588	550	0.7044	-104.2469	-1.8870	0.7077	0.7077	0.7077	0.5652	0.6030	0.5223	8.1947

Framework versions

Transformers 4.42.0
Pytorch 2.3.0+cu121
Datasets 3.2.0
Tokenizers 0.19.1

hZzy/qwen2.5-0.5b-expo-DPO-ES-0.1

qwen2.5-0.5b-expo-DPO-ES-0.1

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
qwen2.5-0.5b-expo-DPO-ES-0.1