This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 0.4658
Objective: 0.4672
Logp Accuracy: 0.5383
Log Diff Policy: 1.7383
Chosen Logps: -87.9379
Rejected Logps: -89.6762
Logits: -2.1598

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.2
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Logits
0.5851	0.0758	50	0.5116	0.5083	0.5176	0.4092	-94.0108	-94.4201	-2.1991
0.5966	0.1517	100	0.5034	0.4990	0.5176	0.5755	-93.3520	-93.9275	-2.1967
0.6151	0.2275	150	0.4887	0.4871	0.5243	0.9455	-92.3973	-93.3428	-2.1785
0.535	0.3033	200	0.4802	0.4802	0.5294	1.1981	-90.9496	-92.1477	-2.1920
0.5207	0.3792	250	0.4757	0.4777	0.5333	1.3292	-92.1322	-93.4614	-2.2157
0.5078	0.4550	300	0.4723	0.4743	0.5341	1.4968	-90.7397	-92.2365	-2.2200
0.4984	0.5308	350	0.4688	0.4694	0.5324	1.5296	-90.5128	-92.0423	-2.2029
0.47	0.6067	400	0.4664	0.4678	0.5352	1.6503	-91.2065	-92.8568	-2.1720
0.4747	0.6825	450	0.4641	0.4656	0.5336	1.5967	-89.2100	-90.8067	-2.1762
0.5021	0.7583	500	0.4733	0.4756	0.5338	1.6804	-86.5736	-88.2540	-2.1824
0.4333	0.8342	550	0.4653	0.4679	0.5380	1.7131	-87.7538	-89.4669	-2.1945
0.4559	0.9100	600	0.4618	0.4649	0.5324	1.5970	-89.3538	-90.9508	-2.1806
0.494	0.9858	650	0.4639	0.4652	0.5355	1.7384	-90.7586	-92.4970	-2.1616

Framework versions

PEFT 0.11.1
Transformers 4.45.2
Pytorch 2.5.1+cu124
Datasets 3.5.0
Tokenizers 0.20.3

hZzy/mistral-7b-expo-7b-L2EXPO-25-last-try-3

mistral-7b-expo-7b-L2EXPO-25-last-try-3

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-L2EXPO-25-last-try-3