This model is a fine-tuned version of hZzy/mistral-7b-sft-25-1 on the hZzy/direction_right2 dataset. It achieves the following results on the evaluation set:

Loss: 1.8510
Objective: 1.7986
Reward Accuracy: 0.6560
Logp Accuracy: 0.6516
Log Diff Policy: 22.4565
Chosen Logps: -204.6168
Rejected Logps: -227.0733
Chosen Rewards: -1.0993
Rejected Rewards: -1.3201
Logits: -1.9099

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 3
eval_batch_size: 3
seed: 42
distributed_type: multi-GPU
num_devices: 3
gradient_accumulation_steps: 12
total_train_batch_size: 108
total_eval_batch_size: 9
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Objective	Reward Accuracy	Logp Accuracy	Log Diff Policy	Chosen Logps	Rejected Logps	Chosen Rewards	Rejected Rewards	Logits
3.0061	0.0758	50	2.8218	2.7372	0.5305	0.5187	0.4617	-92.3331	-92.7948	0.0235	0.0227	-2.1836
3.3194	0.1517	100	2.7832	2.6981	0.5738	0.5274	1.1360	-91.6138	-92.7498	0.0307	0.0231	-2.2099
3.9881	0.2275	150	2.5939	2.5209	0.5727	0.5610	3.8041	-122.6689	-126.4730	-0.2799	-0.3141	-1.8826
2.8978	0.3033	200	2.2837	2.2552	0.5889	0.5881	10.2398	-175.4421	-185.6819	-0.8076	-0.9062	-1.8540
2.6071	0.3792	250	2.1691	2.1567	0.6054	0.6099	14.4500	-155.2871	-169.7371	-0.6060	-0.7467	-1.8919
2.523	0.4550	300	2.0533	2.0431	0.6292	0.6272	17.7341	-198.5022	-216.2363	-1.0382	-1.2117	-1.9566
2.7499	0.5308	350	2.0296	2.0050	0.6337	0.6351	20.5639	-161.9380	-182.5019	-0.6726	-0.8744	-1.8902
2.0013	0.6067	400	1.9976	1.9548	0.6443	0.6395	18.9543	-180.1652	-199.1196	-0.8548	-1.0406	-1.7828
1.9925	0.6825	450	1.9300	1.9017	0.6544	0.6468	22.0320	-182.8835	-204.9156	-0.8820	-1.0985	-1.8294
2.2763	0.7583	500	1.9067	1.8963	0.6499	0.6510	23.9211	-195.5258	-219.4469	-1.0084	-1.2438	-1.9589
1.7949	0.8342	550	1.8775	1.8589	0.6454	0.6482	24.0587	-217.0506	-241.1093	-1.2237	-1.4604	-1.9116
2.0691	0.9100	600	1.8616	1.8308	0.6563	0.6516	24.8101	-203.3374	-228.1475	-1.0866	-1.3308	-1.9567
2.1539	0.9858	650	1.8447	1.7896	0.6597	0.6560	24.2729	-185.9319	-210.2048	-0.9125	-1.1514	-1.9038

Framework versions

PEFT 0.11.1
Transformers 4.42.0
Pytorch 2.6.0+cu124
Datasets 3.2.0
Tokenizers 0.19.1

hZzy/mistral-7b-expo-7b-L2EXPO-25-last-4

mistral-7b-expo-7b-L2EXPO-25-last-4

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

hZzy/
mistral-7b-expo-7b-L2EXPO-25-last-4