model_hh_usp3_dpo5 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_hh_usp3_dpo5

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_hh_usp3_dpo5

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.6259
Rewards/chosen: -6.2124
Rewards/rejected: -11.7257
Rewards/accuracies: 0.7200
Rewards/margins: 5.5132
Logps/rejected: -138.8036
Logps/chosen: -128.0061
Logits/rejected: -0.4196
Logits/chosen: -0.3295

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.0345	2.67	100	0.8548	-2.8854	-6.4253	0.7400	3.5400	-128.2029	-121.3520	-0.2554	-0.2032
0.012	5.33	200	1.2021	-5.9535	-11.1573	0.7700	5.2039	-137.6669	-127.4882	-0.4920	-0.4136
0.005	8.0	300	1.5640	-2.6489	-7.8179	0.7200	5.1690	-130.9881	-120.8790	-0.2887	-0.2011
0.0	10.67	400	1.6069	-6.1709	-11.6980	0.7200	5.5272	-138.7483	-127.9230	-0.4158	-0.3257
0.0	13.33	500	1.6195	-6.2010	-11.7066	0.7300	5.5057	-138.7655	-127.9832	-0.4176	-0.3276
0.0	16.0	600	1.6237	-6.1966	-11.7012	0.7200	5.5045	-138.7545	-127.9745	-0.4185	-0.3287
0.0	18.67	700	1.6053	-6.1901	-11.7285	0.7200	5.5384	-138.8092	-127.9615	-0.4195	-0.3294
0.0	21.33	800	1.6157	-6.1882	-11.7098	0.7200	5.5216	-138.7718	-127.9577	-0.4195	-0.3293
0.0	24.0	900	1.6297	-6.2118	-11.7270	0.7300	5.5153	-138.8062	-128.0048	-0.4202	-0.3298
0.0	26.67	1000	1.6259	-6.2124	-11.7257	0.7200	5.5132	-138.8036	-128.0061	-0.4196	-0.3295

Framework versions

PEFT 0.10.0
Transformers 4.39.3
Pytorch 2.2.2+cu121
Datasets 2.18.0
Tokenizers 0.15.2