This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 4.2023
Objective: 4.2488
Logp Accuracy: 0.5302
Log Diff Policy: 1.3478
Chosen Logps: -91.7249
Rejected Logps: -93.0727
Logits: -2.1963

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.2
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Logits
4.9919	0.0758	50	4.9760	4.9709	0.5162	0.4112	-94.0221	-94.4334	-2.1986
4.776	0.1517	100	4.8183	4.8138	0.5179	0.5890	-92.3714	-92.9604	-2.1918
4.6242	0.2275	150	4.4994	4.5227	0.5215	0.9245	-93.1799	-94.1044	-2.1806
4.1446	0.3033	200	4.3579	4.3849	0.5296	1.1231	-91.9536	-93.0767	-2.2051
4.0524	0.3792	250	4.3097	4.3450	0.5305	1.2282	-91.9540	-93.1822	-2.2202
4.0649	0.4550	300	4.2852	4.3276	0.5280	1.2893	-91.4984	-92.7877	-2.2246
3.9029	0.5308	350	4.2732	4.3033	0.5296	1.2748	-91.2104	-92.4852	-2.2165
3.7523	0.6067	400	4.2382	4.2802	0.5330	1.3602	-89.9468	-91.3069	-2.1992
3.9188	0.6825	450	4.2282	4.2752	0.5271	1.2815	-89.9520	-91.2335	-2.2066
3.924	0.7583	500	4.2284	4.2881	0.5282	1.3782	-90.7221	-92.1003	-2.1915
3.8564	0.8342	550	4.2347	4.3006	0.5296	1.3164	-90.8981	-92.2144	-2.1985
3.7736	0.9100	600	4.2372	4.3025	0.5291	1.3030	-90.7523	-92.0554	-2.2105
3.9957	0.9858	650	4.2052	4.2543	0.5313	1.3570	-91.9905	-93.3475	-2.2042

Framework versions

PEFT 0.11.1
Transformers 4.45.2
Pytorch 2.5.1+cu124
Datasets 3.5.0
Tokenizers 0.20.3

hZzy/mistral-7b-expo-7b-IPO-25-last-try-3

mistral-7b-expo-7b-IPO-25-last-try-3

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-IPO-25-last-try-3