This model is a fine-tuned version of tsavage68/Na_L3_100steps_1e6rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0000
Rewards/chosen: 1.4781
Rewards/rejected: -9.2489
Rewards/accuracies: 1.0
Rewards/margins: 10.7270
Logps/rejected: -72.3314
Logps/chosen: -19.9630
Logits/rejected: -0.9560
Logits/chosen: -0.8870

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-07
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5536	0.2667	50	0.4820	0.0693	-0.4185	1.0	0.4879	-42.8969	-24.6590	-0.9527	-0.8859
0.0373	0.5333	100	0.0199	0.3933	-3.6217	1.0	4.0149	-53.5739	-23.5793	-0.9535	-0.8872
0.0005	0.8	150	0.0003	1.1386	-7.0944	1.0	8.2331	-65.1499	-21.0947	-0.9552	-0.8874
0.0002	1.0667	200	0.0001	1.2700	-7.7758	1.0	9.0458	-67.4211	-20.6567	-0.9551	-0.8867
0.0001	1.3333	250	0.0001	1.3181	-8.1796	1.0	9.4977	-68.7671	-20.4966	-0.9562	-0.8876
0.0001	1.6	300	0.0001	1.3580	-8.4519	1.0	9.8098	-69.6746	-20.3636	-0.9553	-0.8866
0.0001	1.8667	350	0.0001	1.4034	-8.6459	1.0	10.0493	-70.3215	-20.2121	-0.9543	-0.8856
0.0	2.1333	400	0.0000	1.4148	-8.8379	1.0	10.2528	-70.9616	-20.1740	-0.9550	-0.8861
0.0	2.4	450	0.0000	1.4376	-8.9784	1.0	10.4160	-71.4297	-20.0980	-0.9554	-0.8865
0.0	2.6667	500	0.0000	1.4553	-9.0787	1.0	10.5340	-71.7639	-20.0391	-0.9549	-0.8859
0.0	2.9333	550	0.0000	1.4539	-9.1452	1.0	10.5991	-71.9857	-20.0438	-0.9553	-0.8863
0.0	3.2	600	0.0000	1.4778	-9.1833	1.0	10.6610	-72.1127	-19.9643	-0.9547	-0.8856
0.0	3.4667	650	0.0000	1.4769	-9.2273	1.0	10.7042	-72.2595	-19.9671	-0.9550	-0.8859
0.0	3.7333	700	0.0000	1.4827	-9.2281	1.0	10.7108	-72.2620	-19.9477	-0.9543	-0.8853
0.0	4.0	750	0.0000	1.4809	-9.2416	1.0	10.7224	-72.3069	-19.9539	-0.9557	-0.8868
0.0	4.2667	800	0.0000	1.4745	-9.2504	1.0	10.7249	-72.3365	-19.9751	-0.9548	-0.8858
0.0	4.5333	850	0.0000	1.4780	-9.2471	1.0	10.7251	-72.3253	-19.9635	-0.9557	-0.8867
0.0	4.8	900	0.0000	1.4758	-9.2483	1.0	10.7241	-72.3295	-19.9707	-0.9560	-0.8870
0.0	5.0667	950	0.0000	1.4781	-9.2489	1.0	10.7270	-72.3314	-19.9630	-0.9560	-0.8870
0.0	5.3333	1000	0.0000	1.4781	-9.2489	1.0	10.7270	-72.3314	-19.9630	-0.9560	-0.8870

Framework versions

Transformers 4.44.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.19.1

tsavage68/Na_L3_1000steps_1e7rate_03beta_cSFTDPO

Na_L3_1000steps_1e7rate_03beta_cSFTDPO

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

tsavage68/
Na_L3_1000steps_1e7rate_03beta_cSFTDPO