model_hh_usp2_dpo9 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_hh_usp2_dpo9

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_hh_usp2_dpo9

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.6846
Rewards/chosen: -0.5731
Rewards/rejected: -5.6737
Rewards/accuracies: 0.6400
Rewards/margins: 5.1005
Logps/rejected: -120.8027
Logps/chosen: -111.7147
Logits/rejected: -0.2343
Logits/chosen: -0.2007

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.0325	2.67	100	1.7152	-6.7346	-9.0742	0.6300	2.3396	-124.5811	-118.5607	-0.0481	-0.0442
0.0969	5.33	200	3.2390	-15.5354	-20.2430	0.6400	4.7075	-136.9909	-128.3395	-0.4433	-0.4023
0.0009	8.0	300	3.0813	-4.4428	-8.6631	0.6200	4.2203	-124.1243	-116.0144	-0.5257	-0.4694
0.0	10.67	400	2.6646	-0.4885	-5.6039	0.6300	5.1153	-120.7252	-111.6207	-0.2334	-0.1998
0.0	13.33	500	2.6520	-0.5356	-5.6689	0.6400	5.1333	-120.7974	-111.6730	-0.2340	-0.2002
0.0	16.0	600	2.6313	-0.5224	-5.6829	0.6300	5.1605	-120.8130	-111.6584	-0.2338	-0.2001
0.0	18.67	700	2.6655	-0.5685	-5.6921	0.6400	5.1236	-120.8232	-111.7095	-0.2342	-0.2004
0.0	21.33	800	2.6412	-0.5360	-5.6727	0.6400	5.1366	-120.8016	-111.6735	-0.2342	-0.2004
0.0	24.0	900	2.6564	-0.5614	-5.6738	0.6400	5.1124	-120.8029	-111.7017	-0.2347	-0.2010
0.0	26.67	1000	2.6846	-0.5731	-5.6737	0.6400	5.1005	-120.8027	-111.7147	-0.2343	-0.2007

Framework versions

PEFT 0.10.0
Transformers 4.39.3
Pytorch 2.2.2+cu121
Datasets 2.18.0
Tokenizers 0.15.2