Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.7055	0.2	25	0.6877	-0.5880	-0.6760	0.5400	0.0880	-19.9066	-19.7102	-1.0695	-1.0683
0.687	0.4	50	0.5510	3.8005	3.0720	0.6000	0.7284	-12.4107	-10.9333	-1.0238	-1.0226
0.4531	0.6	75	0.1502	4.9713	-4.7070	0.9800	9.6782	-27.9686	-8.5917	-1.0329	-1.0318
0.55	0.8	100	0.0005	4.5705	-5.2643	1.0	9.8348	-29.0833	-9.3932	-0.9625	-0.9598
0.0	1.0	125	0.0000	4.8044	-5.8180	1.0	10.6224	-30.1907	-8.9255	-0.9703	-0.9647
0.1601	1.2	150	0.0000	5.0462	-5.7018	1.0	10.7480	-29.9584	-8.4419	-0.9626	-0.9538
0.0	1.4	175	0.0000	5.1329	-6.1463	1.0	11.2792	-30.8473	-8.2684	-0.9678	-0.9570
0.0	1.6	200	0.0000	5.1779	-6.3913	1.0	11.5693	-31.3374	-8.1783	-0.9704	-0.9584
0.0	1.8	225	0.0000	5.2127	-6.6164	1.0	11.8290	-31.7874	-8.1089	-0.9734	-0.9603
0.0	2.0	250	0.0000	5.2438	-6.7691	1.0	12.0129	-32.0928	-8.0465	-0.9748	-0.9610
0.0	2.2	275	0.0000	5.2700	-6.9223	1.0	12.1924	-32.3994	-7.9942	-0.9773	-0.9626
0.0	2.4	300	0.0000	5.3046	-7.0559	1.0	12.3605	-32.6664	-7.9249	-0.9774	-0.9620
0.0	2.6	325	0.0000	5.3317	-7.1477	1.0	12.4794	-32.8500	-7.8708	-0.9791	-0.9634
0.0	2.8	350	0.0000	5.3455	-7.2371	1.0	12.5826	-33.0289	-7.8432	-0.9799	-0.9635
0.0	3.0	375	0.0000	5.3670	-7.3218	1.0	12.6888	-33.1983	-7.8002	-0.9807	-0.9638
0.0	3.2	400	0.0000	5.3762	-7.4179	1.0	12.7941	-33.3904	-7.7818	-0.9806	-0.9632
0.0	3.4	425	0.0000	5.4071	-7.4781	1.0	12.8852	-33.5110	-7.7200	-0.9825	-0.9651
0.0	3.6	450	0.0000	5.4188	-7.5434	1.0	12.9622	-33.6416	-7.6966	-0.9835	-0.9655
0.0	3.8	475	0.0000	5.4263	-7.6044	1.0	13.0307	-33.7634	-7.6816	-0.9837	-0.9655
0.0	4.0	500	0.0000	5.4259	-7.6600	1.0	13.0859	-33.8747	-7.6824	-0.9841	-0.9656
0.0	4.2	525	0.0000	5.4441	-7.7026	1.0	13.1467	-33.9600	-7.6460	-0.9840	-0.9652
0.0	4.4	550	0.0000	5.4553	-7.7278	1.0	13.1832	-34.0104	-7.6235	-0.9854	-0.9666
0.0	4.6	575	0.0000	5.4630	-7.7823	1.0	13.2453	-34.1192	-7.6081	-0.9852	-0.9662
0.0	4.8	600	0.0000	5.4630	-7.8281	1.0	13.2911	-34.2109	-7.6082	-0.9865	-0.9673
0.0	5.0	625	0.0000	5.4702	-7.8531	1.0	13.3233	-34.2609	-7.5939	-0.9865	-0.9672
0.0	5.2	650	0.0000	5.4827	-7.8764	1.0	13.3591	-34.3075	-7.5687	-0.9853	-0.9659
0.0	5.4	675	0.0000	5.4842	-7.9006	1.0	13.3848	-34.3559	-7.5659	-0.9859	-0.9665
0.0	5.6	700	0.0000	5.4900	-7.9155	1.0	13.4055	-34.3857	-7.5543	-0.9864	-0.9669
0.0	5.8	725	0.0000	5.4865	-7.9426	1.0	13.4291	-34.4398	-7.5612	-0.9860	-0.9664
0.0	6.0	750	0.0000	5.4953	-7.9503	1.0	13.4455	-34.4552	-7.5437	-0.9859	-0.9663
0.0	6.2	775	0.0000	5.4917	-7.9644	1.0	13.4561	-34.4836	-7.5509	-0.9859	-0.9663
0.0	6.4	800	0.0000	5.5003	-7.9640	1.0	13.4642	-34.4826	-7.5337	-0.9853	-0.9657
0.0	6.6	825	0.0000	5.4953	-7.9776	1.0	13.4729	-34.5099	-7.5436	-0.9867	-0.9670
0.0	6.8	850	0.0000	5.4915	-7.9747	1.0	13.4662	-34.5041	-7.5513	-0.9870	-0.9673
0.0	7.0	875	0.0000	5.4933	-7.9815	1.0	13.4748	-34.5177	-7.5476	-0.9870	-0.9675
0.0	7.2	900	0.0000	5.4929	-7.9862	1.0	13.4790	-34.5270	-7.5485	-0.9873	-0.9675
0.0	7.4	925	0.0000	5.4931	-7.9774	1.0	13.4705	-34.5095	-7.5480	-0.9870	-0.9673
0.0	7.6	950	0.0000	5.4967	-7.9805	1.0	13.4772	-34.5156	-7.5408	-0.9869	-0.9672
0.0	7.8	975	0.0000	5.4974	-7.9781	1.0	13.4754	-34.5108	-7.5395	-0.9869	-0.9672
0.0	8.0	1000	0.0000	5.4974	-7.9781	1.0	13.4754	-34.5108	-7.5395	-0.9869	-0.9672

Framework versions

Transformers 4.40.2
Pytorch 2.0.0+cu117
Datasets 2.19.1
Tokenizers 0.19.1

tsavage68/Transaminitis_L3_1000steps_1e6rate_05beta_CSFTDPO

Transaminitis_L3_1000steps_1e6rate_05beta_CSFTDPO

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

tsavage68/
Transaminitis_L3_1000steps_1e6rate_05beta_CSFTDPO