This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 0.6002
Objective: 0.6139
Logp Accuracy: 0.6636
Log Diff Policy: 43.6726
Chosen Logps: -309.4410
Rejected Logps: -353.1136
Logits: -1.3787

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.2
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Logits
0.693	0.0758	50	0.6930	0.6930	0.5154	0.4047	-93.9515	-94.3562	-2.1995
0.6917	0.1517	100	0.6921	0.6921	0.5190	0.6108	-92.2034	-92.8142	-2.2036
0.6865	0.2275	150	0.6868	0.6872	0.5366	1.7198	-92.9224	-94.6422	-2.1207
0.6507	0.3033	200	0.6631	0.6684	0.5845	9.5136	-127.5494	-137.0630	-1.8213
0.629	0.3792	250	0.6505	0.6583	0.6035	15.4656	-131.4656	-146.9312	-1.8424
0.634	0.4550	300	0.6336	0.6415	0.6244	23.3148	-187.6798	-210.9946	-1.6750
0.5837	0.5308	350	0.6326	0.6470	0.6331	32.9779	-242.8130	-275.7909	-1.6081
0.5783	0.6067	400	0.6269	0.6363	0.6451	32.5418	-177.1183	-209.6601	-1.7388
0.5749	0.6825	450	0.6155	0.6246	0.6499	36.7054	-217.9877	-254.6931	-1.6474
0.5651	0.7583	500	0.6151	0.6275	0.6527	43.9688	-287.4218	-331.3907	-1.6310
0.5515	0.8342	550	0.6107	0.6214	0.6602	44.2664	-323.9571	-368.2235	-1.4372
0.5467	0.9100	600	0.6016	0.6105	0.6681	43.5348	-248.7065	-292.2413	-1.4585
0.5926	0.9858	650	0.6003	0.6130	0.6653	41.5848	-276.2677	-317.8525	-1.5049

Framework versions

PEFT 0.11.1
Transformers 4.45.2
Pytorch 2.5.1+cu124
Datasets 3.5.0
Tokenizers 0.20.3

hZzy/mistral-7b-expo-7b-DPO-25-last-try-1

mistral-7b-expo-7b-DPO-25-last-try-1

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-DPO-25-last-try-1