model_usp3_dpo9 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_usp3_dpo9

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_usp3_dpo9

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.3840
Rewards/chosen: -5.8994
Rewards/rejected: -15.8549
Rewards/accuracies: 0.75
Rewards/margins: 9.9555
Logps/rejected: -125.7216
Logps/chosen: -114.4451
Logits/rejected: -0.5607
Logits/chosen: -0.5006

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.1194	2.67	100	1.1073	5.0437	2.1933	0.7400	2.8504	-105.6681	-102.2862	0.0014	0.0428
0.0189	5.33	200	2.5034	-3.9384	-11.8385	0.7000	7.9001	-121.2590	-112.2662	-0.7943	-0.7591
0.0521	8.0	300	2.6657	2.8593	-3.0059	0.6700	5.8652	-111.4450	-104.7133	-0.3470	-0.2646
0.0001	10.67	400	2.4434	-6.5026	-16.5073	0.7400	10.0046	-126.4465	-115.1154	-0.5717	-0.5110
0.0	13.33	500	2.3881	-5.9046	-15.8560	0.75	9.9513	-125.7228	-114.4510	-0.5605	-0.5010
0.0	16.0	600	2.3960	-5.9125	-15.8411	0.75	9.9286	-125.7063	-114.4597	-0.5602	-0.5003
0.0	18.67	700	2.3936	-5.8978	-15.8162	0.75	9.9184	-125.6786	-114.4434	-0.5604	-0.5003
0.0	21.33	800	2.3929	-5.9227	-15.8715	0.75	9.9488	-125.7401	-114.4710	-0.5609	-0.5010
0.0	24.0	900	2.3975	-5.9447	-15.8363	0.75	9.8917	-125.7010	-114.4955	-0.5609	-0.5009
0.0	26.67	1000	2.3840	-5.8994	-15.8549	0.75	9.9555	-125.7216	-114.4451	-0.5607	-0.5006

Framework versions

PEFT 0.10.0
Transformers 4.39.3
Pytorch 2.2.2+cu121
Datasets 2.18.0
Tokenizers 0.15.2