This model is a fine-tuned version of tsavage68/Hyponatremia_M2_150steps_1e6rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0000
Rewards/chosen: -1.6269
Rewards/rejected: -29.3999
Rewards/accuracies: 1.0
Rewards/margins: 27.7730
Logps/rejected: -129.6812
Logps/chosen: -39.9809
Logits/rejected: -2.1473
Logits/chosen: -2.1867

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Epoch	Step	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.2667	50	-2.3268	-27.3496	1.0	25.0229	-125.5807	-41.3806	-2.1546	-2.1935
0.5333	100	-2.0801	-28.0698	1.0	25.9898	-127.0211	-40.8872	-2.1528	-2.1920
0.8	150	-1.9471	-28.4315	1.0	26.4844	-127.7445	-40.6212	-2.1520	-2.1911
1.0667	200	-1.8478	-28.6613	1.0	26.8135	-128.2041	-40.4227	-2.1513	-2.1907
1.3333	250	-1.7994	-28.8258	1.0	27.0265	-128.5331	-40.3258	-2.1505	-2.1897
1.6	300	-1.7572	-28.9306	1.0	27.1734	-128.7426	-40.2414	-2.1503	-2.1896
1.8667	350	-1.7112	-29.0453	1.0	27.3341	-128.9720	-40.1494	-2.1510	-2.1902
2.1333	400	-1.6918	-29.1070	1.0	27.4152	-129.0955	-40.1106	-2.1497	-2.1890
2.4	450	-1.6571	-29.1867	1.0	27.5295	-129.2548	-40.0413	-2.1486	-2.1879
2.6667	500	-1.6758	-29.3111	1.0	27.6353	-129.5036	-40.0786	-2.1477	-2.1871
2.9333	550	-1.6493	-29.2708	1.0	27.6214	-129.4230	-40.0257	-2.1478	-2.1872
3.2	600	-1.6166	-29.3299	1.0	27.7133	-129.5412	-39.9603	-2.1476	-2.1870
3.4667	650	-1.6259	-29.3691	1.0	27.7432	-129.6197	-39.9788	-2.1469	-2.1864
3.7333	700	-1.6117	-29.4111	1.0	27.7995	-129.7038	-39.9504	-2.1462	-2.1858
4.0	750	-1.6390	-29.3989	1.0	27.7599	-129.6792	-40.0050	-2.1472	-2.1865
4.2667	800	-1.6061	-29.4262	1.0	27.8201	-129.7338	-39.9392	-2.1473	-2.1868
4.5333	850	-1.6214	-29.4140	1.0	27.7926	-129.7094	-39.9698	-2.1475	-2.1870
4.8	900	-1.6269	-29.3999	1.0	27.7730	-129.6812	-39.9809	-2.1473	-2.1867
5.0667	950	-1.6269	-29.3999	1.0	27.7730	-129.6812	-39.9809	-2.1473	-2.1867
5.3333	1000	-1.6269	-29.3999	1.0	27.7730	-129.6812	-39.9809	-2.1473	-2.1867

Framework versions

Transformers 4.42.4
Pytorch 2.0.0+cu117
Datasets 2.20.0
Tokenizers 0.19.1

tsavage68/Hyponatremia_M2_1000steps_1e5rate_05beta_DPO

Hyponatremia_M2_1000steps_1e5rate_05beta_DPO

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

tsavage68/
Hyponatremia_M2_1000steps_1e5rate_05beta_DPO