model_usp1_dpo5 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_usp1_dpo5

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_usp1_dpo5

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.2734
Rewards/chosen: -2.3548
Rewards/rejected: -7.1210
Rewards/accuracies: 0.6900
Rewards/margins: 4.7662
Logps/rejected: -126.1126
Logps/chosen: -111.5945
Logits/rejected: -0.6605
Logits/chosen: -0.6138

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.0275	2.67	100	0.8121	-5.6383	-8.4322	0.75	2.7939	-128.7351	-118.1615	-0.5793	-0.6303
0.0206	5.33	200	1.5888	-6.3660	-10.1184	0.7300	3.7524	-132.1074	-119.6168	-0.9589	-0.9756
0.0046	8.0	300	1.3556	-7.4924	-13.2011	0.7400	5.7087	-138.2728	-121.8697	-0.9273	-0.9021
0.0	10.67	400	1.2681	-2.2118	-6.9174	0.6700	4.7056	-125.7056	-111.3086	-0.6495	-0.6029
0.0	13.33	500	1.2678	-2.2883	-7.0174	0.6600	4.7291	-125.9055	-111.4616	-0.6552	-0.6085
0.0	16.0	600	1.2844	-2.3230	-7.0735	0.6700	4.7505	-126.0177	-111.5310	-0.6587	-0.6122
0.0	18.67	700	1.2742	-2.3383	-7.1048	0.6800	4.7665	-126.0803	-111.5616	-0.6601	-0.6134
0.0	21.33	800	1.2810	-2.3731	-7.1141	0.6800	4.7409	-126.0988	-111.6312	-0.6607	-0.6139
0.0	24.0	900	1.2683	-2.3582	-7.1234	0.6700	4.7652	-126.1175	-111.6013	-0.6609	-0.6140
0.0	26.67	1000	1.2734	-2.3548	-7.1210	0.6900	4.7662	-126.1126	-111.5945	-0.6605	-0.6138

Framework versions

PEFT 0.10.0
Transformers 4.39.3
Pytorch 2.2.2+cu121
Datasets 2.18.0
Tokenizers 0.15.2