model_shp3_dpo5 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_shp3_dpo5

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_shp3_dpo5

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.1787
Rewards/chosen: -6.1846
Rewards/rejected: -8.5969
Rewards/accuracies: 0.6200
Rewards/margins: 2.4123
Logps/rejected: -271.1182
Logps/chosen: -230.6183
Logits/rejected: -0.9872
Logits/chosen: -0.9880

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.028	2.67	100	1.0860	-2.5770	-3.4358	0.5900	0.8588	-260.7960	-223.4030	-0.8288	-0.8294
0.0224	5.33	200	1.5814	-7.3321	-9.0973	0.5900	1.7652	-272.1189	-232.9132	-0.9907	-0.9251
0.0108	8.0	300	1.6622	-3.3135	-4.7967	0.5700	1.4832	-263.5178	-224.8760	-0.9527	-0.9456
0.0001	10.67	400	2.2015	-6.2017	-8.5794	0.6100	2.3777	-271.0831	-230.6524	-0.9859	-0.9872
0.0	13.33	500	2.1911	-6.1679	-8.5594	0.6200	2.3916	-271.0432	-230.5848	-0.9860	-0.9873
0.0	16.0	600	2.1885	-6.1988	-8.6070	0.6300	2.4081	-271.1382	-230.6467	-0.9867	-0.9877
0.0	18.67	700	2.1971	-6.2183	-8.6010	0.6200	2.3827	-271.1264	-230.6856	-0.9868	-0.9876
0.0	21.33	800	2.1909	-6.1952	-8.5881	0.6300	2.3929	-271.1005	-230.6395	-0.9863	-0.9870
0.0	24.0	900	2.1721	-6.1855	-8.6019	0.6100	2.4164	-271.1281	-230.6200	-0.9871	-0.9878
0.0	26.67	1000	2.1787	-6.1846	-8.5969	0.6200	2.4123	-271.1182	-230.6183	-0.9872	-0.9880

Framework versions

PEFT 0.10.0
Transformers 4.39.1
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2