This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 41.0707
Objective: 41.6895
Logp Accuracy: 0.6342
Log Diff Policy: 10.4127
Chosen Logps: -135.0471
Rejected Logps: -145.4598
Logits: -2.0767

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.2
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Logits
49.9662	0.0758	50	49.9739	49.9694	0.5154	0.4128	-94.0313	-94.4441	-2.1990
49.7291	0.1517	100	49.7995	49.7852	0.5187	0.5970	-91.8025	-92.3994	-2.1862
48.6514	0.2275	150	48.7960	48.8704	0.5327	1.5118	-92.2439	-93.7557	-2.1369
44.0066	0.3033	200	45.2763	45.6633	0.5808	5.4226	-112.1535	-117.5761	-1.9035
42.7721	0.3792	250	44.0720	44.5823	0.5973	7.0156	-110.1814	-117.1970	-1.8638
43.0951	0.4550	300	43.1124	43.7034	0.6127	7.9032	-137.4939	-145.3970	-1.8230
40.1332	0.5308	350	43.0172	43.8942	0.6147	10.4637	-155.2350	-165.6986	-1.8842
39.5136	0.6067	400	42.2942	42.8290	0.6202	9.9044	-116.1799	-126.0843	-1.9302
40.3215	0.6825	450	42.1473	43.0121	0.6275	11.8776	-137.1173	-148.9949	-1.9209
39.833	0.7583	500	42.0706	42.6517	0.6337	12.4018	-119.9330	-132.3348	-2.0385
37.991	0.8342	550	41.4779	42.1021	0.6334	11.5728	-137.8443	-149.4171	-2.0752
38.4834	0.9100	600	41.1538	41.6987	0.6412	11.6922	-124.0182	-135.7104	-2.0150
40.7553	0.9858	650	40.9628	41.5879	0.6398	10.8595	-134.4952	-145.3548	-2.0644

Framework versions

PEFT 0.11.1
Transformers 4.45.2
Pytorch 2.5.1+cu124
Datasets 3.5.0
Tokenizers 0.20.3

hZzy/mistral-7b-expo-7b-IPO-25-last-try-1

mistral-7b-expo-7b-IPO-25-last-try-1

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-IPO-25-last-try-1