model_shp4_dpo9 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_shp4_dpo9

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_shp4_dpo9

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 4.9330
Rewards/chosen: -5.0952
Rewards/rejected: -3.3852
Rewards/accuracies: 0.4400
Rewards/margins: -1.7101
Logps/rejected: -229.9833
Logps/chosen: -243.2094
Logits/rejected: -0.6262
Logits/chosen: -0.6100

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.019	2.67	100	3.1180	-4.1006	-3.1231	0.4500	-0.9774	-229.6921	-242.1042	-0.7573	-0.7550
0.1473	5.33	200	3.3714	-1.0308	-1.7817	0.4800	0.7508	-228.2016	-238.6933	-0.6529	-0.6671
0.0024	8.0	300	4.5106	-1.0334	0.7012	0.4200	-1.7345	-225.4429	-238.6962	-0.6564	-0.6438
0.0	10.67	400	5.5670	-10.8604	-9.4760	0.4500	-1.3844	-236.7508	-249.6151	-0.6527	-0.6452
0.0	13.33	500	4.9503	-5.0348	-3.3246	0.4400	-1.7103	-229.9160	-243.1422	-0.6262	-0.6099
0.0	16.0	600	4.9114	-5.0555	-3.4112	0.4400	-1.6443	-230.0122	-243.1652	-0.6265	-0.6103
0.0	18.67	700	4.9251	-5.0566	-3.3811	0.4400	-1.6755	-229.9788	-243.1664	-0.6262	-0.6101
0.0	21.33	800	4.9218	-5.0650	-3.3763	0.4400	-1.6887	-229.9734	-243.1758	-0.6259	-0.6098
0.0	24.0	900	4.9342	-5.1119	-3.3937	0.4500	-1.7181	-229.9928	-243.2278	-0.6259	-0.6098
0.0	26.67	1000	4.9330	-5.0952	-3.3852	0.4400	-1.7101	-229.9833	-243.2094	-0.6262	-0.6100

Framework versions

PEFT 0.10.0
Transformers 4.39.1
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2