This model is a fine-tuned version of tsavage68/mistralit2_1000_STEPS_5e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.9166
Rewards/chosen: -3.7686
Rewards/rejected: -6.1250
Rewards/accuracies: 0.6659
Rewards/margins: 2.3564
Logps/rejected: -87.8066
Logps/chosen: -61.3577
Logits/rejected: -1.6465
Logits/chosen: -1.6475

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5681	0.0977	50	0.5897	-0.2848	-0.7807	0.6220	0.4959	-34.3643	-26.5198	-2.0822	-2.0819
0.5454	0.1953	100	0.7078	-1.8828	-2.8036	0.6198	0.9208	-54.5930	-42.5001	-1.8193	-1.8193
0.7509	0.2930	150	0.6446	-0.7755	-1.2878	0.6022	0.5123	-39.4349	-31.4272	-2.3220	-2.3221
0.5784	0.3906	200	0.6519	-1.2019	-1.8232	0.6132	0.6213	-44.7889	-35.6908	-2.2690	-2.2690
0.7042	0.4883	250	0.6671	0.2266	-0.1145	0.5560	0.3411	-27.7019	-21.4055	-2.6333	-2.6332
0.5541	0.5859	300	0.6786	-1.5906	-2.3448	0.5868	0.7542	-50.0051	-39.5776	-2.7458	-2.7459
0.5497	0.6836	350	0.7057	-1.7745	-2.5281	0.6154	0.7537	-51.8382	-41.4164	-2.3898	-2.3902
0.7584	0.7812	400	0.6186	-0.5869	-1.5391	0.6637	0.9522	-41.9478	-29.5405	-2.4709	-2.4713
0.5936	0.8789	450	0.6546	-1.3995	-2.1895	0.6176	0.7901	-48.4523	-37.6665	-2.4013	-2.4017
0.4586	0.9766	500	0.6362	-1.1700	-2.0896	0.6330	0.9196	-47.4530	-35.3713	-2.0695	-2.0701
0.1066	1.0742	550	0.8659	-3.6103	-5.5577	0.6593	1.9474	-82.1337	-59.7750	-1.2953	-1.2968
0.1745	1.1719	600	0.8617	-3.4521	-5.5739	0.6374	2.1218	-82.2959	-58.1929	-1.4289	-1.4301
0.2043	1.2695	650	0.8388	-3.1557	-5.3780	0.6615	2.2223	-80.3369	-55.2288	-1.5113	-1.5124
0.391	1.3672	700	0.9441	-3.8602	-6.1157	0.6659	2.2555	-87.7136	-62.2736	-1.6592	-1.6602
0.0658	1.4648	750	0.9386	-3.7565	-6.1054	0.6549	2.3489	-87.6110	-61.2367	-1.6260	-1.6270
0.1568	1.5625	800	0.9240	-3.7963	-6.1474	0.6615	2.3510	-88.0308	-61.6352	-1.6191	-1.6201
0.19	1.6602	850	0.9144	-3.7476	-6.0958	0.6659	2.3482	-87.5153	-61.1479	-1.6336	-1.6346
0.1631	1.7578	900	0.9176	-3.7680	-6.1169	0.6681	2.3489	-87.7256	-61.3513	-1.6457	-1.6466
0.1537	1.8555	950	0.9149	-3.7653	-6.1210	0.6659	2.3558	-87.7673	-61.3244	-1.6467	-1.6476
0.134	1.9531	1000	0.9166	-3.7686	-6.1250	0.6659	2.3564	-87.8066	-61.3577	-1.6465	-1.6475

Framework versions

Transformers 4.40.1
Pytorch 2.0.0+cu117
Datasets 2.19.0
Tokenizers 0.19.1

tsavage68/Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT

Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

tsavage68/
Mistral2_1000_STEPS_01beta_1e6rate_CDPOSFT