This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 8.3821
Objective: 8.4909
Logp Accuracy: 0.5487
Log Diff Policy: 2.3898
Chosen Logps: -93.2524
Rejected Logps: -95.6422
Logits: -2.1573

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.2
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Logits
9.9587	0.0758	50	9.9745	9.9675	0.5168	0.4147	-93.9780	-94.3926	-2.1984
9.7294	0.1517	100	9.7902	9.7812	0.5179	0.6025	-91.8236	-92.4261	-2.1967
9.1671	0.2275	150	9.2169	9.2133	0.5305	1.2712	-88.9166	-90.1878	-2.1464
8.6338	0.3033	200	8.8685	8.9130	0.5389	1.6793	-91.6431	-93.3224	-2.1369
8.3244	0.3792	250	8.7069	8.7702	0.5470	1.9875	-93.4366	-95.4241	-2.1516
8.2365	0.4550	300	8.5893	8.6560	0.5501	2.1964	-90.8195	-93.0159	-2.2165
7.8333	0.5308	350	8.5425	8.6419	0.5450	2.1700	-91.8347	-94.0047	-2.2345
7.7736	0.6067	400	8.4782	8.5691	0.5506	2.3681	-89.5435	-91.9116	-2.2336
7.9329	0.6825	450	8.4605	8.5963	0.5459	2.2430	-88.4759	-90.7189	-2.2269
7.8527	0.7583	500	8.4489	8.5705	0.5475	2.3799	-89.6442	-92.0242	-2.2000
7.7046	0.8342	550	8.4520	8.5831	0.5475	2.3762	-90.3442	-92.7204	-2.2031
7.6536	0.9100	600	8.4253	8.5800	0.5447	2.2408	-90.3569	-92.5977	-2.1862
8.1434	0.9858	650	8.3766	8.4881	0.5481	2.4320	-94.5504	-96.9824	-2.1669

Framework versions

PEFT 0.11.1
Transformers 4.45.2
Pytorch 2.5.1+cu124
Datasets 3.5.0
Tokenizers 0.20.3

hZzy/mistral-7b-expo-7b-IPO-25-last-try-2

mistral-7b-expo-7b-IPO-25-last-try-2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-IPO-25-last-try-2