This model is a fine-tuned version of tsavage68/Transaminitis_L3_1000rate_1e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.6914
Rewards/chosen: -15.4983
Rewards/rejected: -15.7754
Rewards/accuracies: 0.3000
Rewards/margins: 0.2771
Logps/rejected: -50.1055
Logps/chosen: -49.5308
Logits/rejected: -0.7536
Logits/chosen: -0.7536

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
1.3308	0.2	25	1.4218	-5.1457	-5.2961	0.5400	0.1503	-29.1468	-28.8257	-0.7892	-0.7880
1.1498	0.4	50	0.7304	-4.8999	-4.8425	0.4000	-0.0574	-28.2397	-28.3340	-2.1796	-2.1797
1.2832	0.6	75	0.9255	-1.6896	-4.2819	0.6300	2.5923	-27.1184	-21.9133	-1.0885	-1.0850
2.8764	0.8	100	3.8444	-19.0391	-19.6042	0.5400	0.5651	-57.7631	-56.6124	-0.1327	-0.1327
0.8442	1.0	125	0.7901	-16.2193	-16.1877	0.5400	-0.0316	-50.9301	-50.9727	-0.7765	-0.7765
0.7539	1.2	150	0.8102	-15.9518	-15.8097	0.4600	-0.1421	-50.1741	-50.4379	-0.9130	-0.9130
0.7462	1.4	175	0.7415	-16.1492	-16.0632	0.4100	-0.0860	-50.6811	-50.8325	-0.8303	-0.8304
0.7363	1.6	200	0.7404	-16.2295	-16.1487	0.4300	-0.0808	-50.8521	-50.9933	-0.8473	-0.8473
0.7666	1.8	225	0.8203	-16.1693	-16.0294	0.4600	-0.1399	-50.6135	-50.8729	-0.9939	-0.9939
0.7639	2.0	250	0.8115	-16.1899	-16.0490	0.4600	-0.1409	-50.6527	-50.9140	-0.8241	-0.8241
0.7153	2.2	275	0.7477	-16.3133	-16.2548	0.5200	-0.0585	-51.0642	-51.1609	-0.7566	-0.7566
0.8015	2.4	300	0.7461	-16.9989	-16.9443	0.5200	-0.0546	-52.4434	-52.5321	-0.7484	-0.7484
0.7741	2.6	325	0.8205	-16.7965	-16.6632	0.4600	-0.1333	-51.8812	-52.1273	-0.8410	-0.8410
0.8986	2.8	350	0.7380	-18.5683	-18.4872	0.3000	-0.0811	-55.5292	-55.6709	-1.2363	-1.2363
0.849	3.0	375	2.3943	-12.5963	-12.1503	0.4600	-0.4460	-42.8553	-43.7269	-0.4070	-0.4065
0.8088	3.2	400	0.7402	-15.8638	-15.7863	0.4600	-0.0775	-50.1272	-50.2618	-0.6327	-0.6327
0.8743	3.4	425	0.7330	-18.1568	-18.0906	0.4100	-0.0662	-54.7359	-54.8479	-1.1648	-1.1647
0.7984	3.6	450	0.7252	-17.1837	-17.1365	0.3300	-0.0472	-52.8276	-52.9015	-1.0496	-1.0496
0.7968	3.8	475	0.8038	-15.3963	-15.3324	0.5400	-0.0639	-49.2195	-49.3268	-0.5901	-0.5901
0.6856	4.0	500	0.7152	-15.3527	-15.4448	0.5300	0.0921	-49.4443	-49.2396	-0.6386	-0.6386
0.7167	4.2	525	0.7150	-15.4946	-15.5966	0.5100	0.1019	-49.7478	-49.5235	-0.6307	-0.6307
0.6039	4.4	550	0.7637	-15.4627	-15.6191	0.5400	0.1563	-49.7928	-49.4597	-0.7779	-0.7779
0.7734	4.6	575	0.7098	-15.4720	-15.6304	0.5300	0.1584	-49.8155	-49.4783	-0.7391	-0.7391
0.6561	4.8	600	0.7003	-15.6141	-15.8015	0.5100	0.1874	-50.1577	-49.7625	-0.7691	-0.7691
0.8328	5.0	625	0.6902	-15.6776	-15.8918	0.2800	0.2141	-50.3382	-49.8894	-0.7913	-0.7913
0.6256	5.2	650	0.6963	-15.6139	-15.8252	0.4800	0.2113	-50.2051	-49.7620	-0.7909	-0.7909
0.7336	5.4	675	0.7511	-15.6031	-15.7883	0.5400	0.1852	-50.1313	-49.7403	-0.7741	-0.7741
0.6527	5.6	700	0.7877	-15.3869	-15.6214	0.5400	0.2345	-49.7974	-49.3080	-0.7426	-0.7426
0.705	5.8	725	0.6894	-15.4753	-15.7539	0.2900	0.2786	-50.0625	-49.4848	-0.7283	-0.7283
0.7304	6.0	750	0.6899	-15.4744	-15.7563	0.2600	0.2819	-50.0674	-49.4830	-0.7329	-0.7329
0.7198	6.2	775	0.6920	-15.5016	-15.7713	0.3800	0.2697	-50.0972	-49.5374	-0.7513	-0.7513
0.7129	6.4	800	0.6908	-15.5077	-15.7810	0.3200	0.2733	-50.1167	-49.5497	-0.7483	-0.7483
0.6531	6.6	825	0.6900	-15.4995	-15.7803	0.2900	0.2807	-50.1153	-49.5333	-0.7526	-0.7526
0.7044	6.8	850	0.6918	-15.4889	-15.7660	0.3600	0.2771	-50.0868	-49.5121	-0.7521	-0.7520
0.6293	7.0	875	0.6914	-15.4926	-15.7693	0.3700	0.2766	-50.0933	-49.5195	-0.7537	-0.7537
0.7101	7.2	900	0.6905	-15.4995	-15.7785	0.2800	0.2789	-50.1116	-49.5333	-0.7528	-0.7528
0.6389	7.4	925	0.6913	-15.4980	-15.7753	0.3300	0.2772	-50.1052	-49.5303	-0.7532	-0.7532
0.6333	7.6	950	0.6907	-15.4984	-15.7771	0.3200	0.2786	-50.1088	-49.5310	-0.7534	-0.7534
0.6491	7.8	975	0.6912	-15.4974	-15.7749	0.3200	0.2775	-50.1045	-49.5291	-0.7534	-0.7534
0.6433	8.0	1000	0.6914	-15.4983	-15.7754	0.3000	0.2771	-50.1055	-49.5308	-0.7536	-0.7536

Framework versions

Transformers 4.40.2
Pytorch 2.0.0+cu117
Datasets 2.19.1
Tokenizers 0.19.1

RichardErkhov/tsavage68_-_Transaminitis_L3_1000steps_1e5rate_05beta_CSFTDPO-4bits

Original model description:

Transaminitis_L3_1000steps_1e5rate_05beta_CSFTDPO

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

RichardErkhov/
tsavage68_-_Transaminitis_L3_1000steps_1e5rate_05beta_CSFTDPO-4bits