This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 0.6196
Objective: 0.6335
Logp Accuracy: 0.5814
Log Diff Policy: 4.6358
Chosen Logps: -93.8576
Rejected Logps: -98.4934
Logits: -2.1076

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.2
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Logits
0.6925	0.0758	50	0.6922	0.6920	0.5168	0.4099	-94.0256	-94.4355	-2.1986
0.6833	0.1517	100	0.6859	0.6854	0.5187	0.5632	-93.6329	-94.1960	-2.2043
0.6724	0.2275	150	0.6710	0.6719	0.5254	1.0307	-92.9415	-93.9723	-2.1765
0.6358	0.3033	200	0.6555	0.6590	0.5391	1.6592	-91.3758	-93.0350	-2.1713
0.6207	0.3792	250	0.6446	0.6507	0.5495	2.3176	-91.7200	-94.0377	-2.1658
0.6135	0.4550	300	0.6364	0.6439	0.5612	2.8332	-91.5311	-94.3643	-2.2105
0.5865	0.5308	350	0.6323	0.6405	0.5593	3.2517	-94.9725	-98.2242	-2.2530
0.5621	0.6067	400	0.6303	0.6424	0.5719	3.9721	-92.3822	-96.3543	-2.2063
0.5771	0.6825	450	0.6254	0.6386	0.5738	3.8779	-90.6563	-94.5341	-2.2001
0.5812	0.7583	500	0.6296	0.6416	0.5786	4.4883	-89.0386	-93.5268	-2.2234
0.5554	0.8342	550	0.6299	0.6400	0.5808	4.6942	-90.7880	-95.4822	-2.1917
0.5562	0.9100	600	0.6204	0.6321	0.5766	4.4140	-92.4615	-96.8755	-2.1767
0.5975	0.9858	650	0.6212	0.6332	0.5817	4.6106	-97.9449	-102.5555	-2.1148

Framework versions

PEFT 0.11.1
Transformers 4.45.2
Pytorch 2.5.1+cu124
Datasets 3.5.0
Tokenizers 0.20.3

hZzy/mistral-7b-expo-7b-DPO-25-last-try-3

mistral-7b-expo-7b-DPO-25-last-try-3

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-DPO-25-last-try-3