This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 0.4604
Objective: 0.4615
Logp Accuracy: 0.5626
Log Diff Policy: 3.2049
Chosen Logps: -87.0040
Rejected Logps: -90.2089
Logits: -2.1619

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.2
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Logits
0.5866	0.0758	50	0.5115	0.5084	0.5165	0.4067	-94.0612	-94.4680	-2.1999
0.6033	0.1517	100	0.5066	0.5039	0.5176	0.5724	-92.5771	-93.1496	-2.1854
0.6226	0.2275	150	0.4953	0.4934	0.5263	1.1162	-92.1086	-93.2248	-2.1539
0.5432	0.3033	200	0.4830	0.4821	0.5366	1.7538	-88.0451	-89.7988	-2.1780
0.5274	0.3792	250	0.4787	0.4796	0.5489	2.1864	-92.7989	-94.9853	-2.1980
0.5128	0.4550	300	0.4737	0.4747	0.5537	2.5705	-90.6816	-93.2521	-2.2376
0.5016	0.5308	350	0.4686	0.4685	0.5543	2.7334	-89.2678	-92.0011	-2.2419
0.4821	0.6067	400	0.4655	0.4661	0.5584	2.9188	-90.9047	-93.8235	-2.2273
0.4797	0.6825	450	0.4639	0.4658	0.5601	2.9840	-88.9118	-91.8957	-2.2192
0.5063	0.7583	500	0.4691	0.4691	0.5604	3.1099	-88.1264	-91.2364	-2.2207
0.4412	0.8342	550	0.4682	0.4696	0.5635	3.2183	-88.1349	-91.3532	-2.2235
0.452	0.9100	600	0.4602	0.4616	0.5582	3.0475	-89.2814	-92.3289	-2.2130
0.4949	0.9858	650	0.4621	0.4634	0.5618	3.1922	-90.6560	-93.8482	-2.1648

Framework versions

PEFT 0.11.1
Transformers 4.45.2
Pytorch 2.5.1+cu124
Datasets 3.5.0
Tokenizers 0.20.3

hZzy/mistral-7b-expo-7b-L2EXPO-25-last-try-2

mistral-7b-expo-7b-L2EXPO-25-last-try-2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-L2EXPO-25-last-try-2