This model is a fine-tuned version of tsavage68/MedQA_L3_1000steps_1e6rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.9802
Rewards/chosen: -1.8607
Rewards/rejected: -1.7391
Rewards/accuracies: 0.4505
Rewards/margins: -0.1215
Logps/rejected: -51.2462
Logps/chosen: -49.9353
Logits/rejected: -0.2251
Logits/chosen: -0.2248

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.671	0.0489	50	1.6433	-6.4141	-6.3515	0.4747	-0.0626	-97.3700	-95.4696	-0.6453	-0.6453
1.0504	0.0977	100	0.8270	-1.6657	-1.8409	0.5385	0.1752	-52.2642	-47.9860	-1.0550	-1.0545
1.4654	0.1466	150	0.9969	-1.4406	-1.2778	0.4264	-0.1627	-46.6333	-45.7340	-0.2863	-0.2860
1.2453	0.1954	200	1.6314	-5.7863	-5.5157	0.4462	-0.2706	-89.0113	-89.1912	1.2703	1.2702
1.0999	0.2443	250	1.0650	-2.0798	-1.9143	0.4549	-0.1655	-52.9977	-52.1260	-0.3259	-0.3258
1.6167	0.2931	300	1.0970	-2.8882	-2.6210	0.4374	-0.2672	-60.0648	-60.2105	-0.5895	-0.5898
1.251	0.3420	350	1.0338	-1.6529	-1.4770	0.4374	-0.1759	-48.6251	-47.8575	-0.1797	-0.1796
1.3582	0.3908	400	1.0344	-2.2844	-2.1347	0.4505	-0.1498	-55.2016	-54.1729	-0.3671	-0.3669
1.3581	0.4397	450	1.0581	-2.2666	-2.0185	0.4286	-0.2481	-54.0398	-53.9945	-0.4232	-0.4233
1.398	0.4885	500	1.0994	-3.1646	-2.9353	0.4110	-0.2293	-63.2075	-62.9742	-0.6033	-0.6033
1.2895	0.5374	550	1.0714	-2.3198	-2.0945	0.4352	-0.2252	-54.8002	-54.5263	-0.2667	-0.2665
1.2884	0.5862	600	1.3491	-5.2367	-5.0465	0.4264	-0.1902	-84.3200	-83.6955	-0.5133	-0.5133
0.9758	0.6351	650	1.0323	-1.9192	-1.7312	0.4396	-0.1880	-51.1668	-50.5202	-0.2364	-0.2363
0.9671	0.6839	700	1.0307	-1.8280	-1.6474	0.4484	-0.1806	-50.3290	-49.6088	-0.2707	-0.2706
1.1016	0.7328	750	1.0113	-1.9758	-1.8284	0.4374	-0.1474	-52.1388	-51.0861	-0.2470	-0.2469
1.0075	0.7816	800	0.9896	-2.0327	-1.9017	0.4462	-0.1310	-52.8716	-51.6551	-0.2568	-0.2566
1.3333	0.8305	850	0.9832	-1.8654	-1.7449	0.4484	-0.1205	-51.3041	-49.9827	-0.2344	-0.2341
1.0175	0.8793	900	0.9806	-1.8682	-1.7465	0.4527	-0.1217	-51.3197	-50.0107	-0.2269	-0.2267
1.1061	0.9282	950	0.9806	-1.8612	-1.7388	0.4462	-0.1224	-51.2424	-49.9402	-0.2250	-0.2248
0.8508	0.9770	1000	0.9802	-1.8607	-1.7391	0.4505	-0.1215	-51.2462	-49.9353	-0.2251	-0.2248

Framework versions

Transformers 4.41.1
Pytorch 2.0.0+cu117
Datasets 2.19.1
Tokenizers 0.19.1

tsavage68/MedQA_L3_1000steps_1e5rate_01beta_CSFTDPO

MedQA_L3_1000steps_1e5rate_01beta_CSFTDPO

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

tsavage68/
MedQA_L3_1000steps_1e5rate_01beta_CSFTDPO