model_shp2_dpo5 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_shp2_dpo5

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_shp2_dpo5

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.1695
Rewards/chosen: -3.4447
Rewards/rejected: -4.4007
Rewards/accuracies: 0.5400
Rewards/margins: 0.9560
Logps/rejected: -245.2697
Logps/chosen: -255.8932
Logits/rejected: -0.8140
Logits/chosen: -0.7999

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.0371	2.67	100	1.6792	-0.5920	-0.6679	0.4600	0.0759	-237.8042	-250.1878	-0.5568	-0.5337
0.0071	5.33	200	1.8321	0.5323	0.0212	0.5500	0.5112	-236.4261	-247.9392	-0.7524	-0.7416
0.0001	8.0	300	2.1597	-2.9208	-3.8736	0.5400	0.9528	-244.2156	-254.8454	-0.8222	-0.8092
0.0	10.67	400	2.1619	-3.3914	-4.3596	0.5300	0.9682	-245.1877	-255.7867	-0.8137	-0.8004
0.0	13.33	500	2.1533	-3.3951	-4.3755	0.5300	0.9804	-245.2194	-255.7940	-0.8134	-0.7998
0.0	16.0	600	2.1833	-3.4274	-4.3755	0.5300	0.9480	-245.2194	-255.8587	-0.8142	-0.8001
0.0	18.67	700	2.1523	-3.4138	-4.3824	0.5300	0.9686	-245.2332	-255.8314	-0.8134	-0.7991
0.0	21.33	800	2.1568	-3.4182	-4.3819	0.5300	0.9637	-245.2321	-255.8403	-0.8134	-0.7993
0.0	24.0	900	2.1621	-3.4517	-4.3884	0.5300	0.9367	-245.2452	-255.9073	-0.8143	-0.8002
0.0	26.67	1000	2.1695	-3.4447	-4.4007	0.5400	0.9560	-245.2697	-255.8932	-0.8140	-0.7999

Framework versions

PEFT 0.10.0
Transformers 4.39.1
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2