model_shp4_dpo5 – AI Model by guoyu-zhang | AlphaNeural AI

You can deploy this model and start earning money today!

guoyu-zhang/
model_shp4_dpo5

like

0

generated_from_trainer

meta-llama/Llama-2-7b-chat-hf

Views

No views yet

model_shp4_dpo5

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.6900
Rewards/chosen: -4.3601
Rewards/rejected: -4.8463
Rewards/accuracies: 0.5100
Rewards/margins: 0.4862
Logps/rejected: -247.0597
Logps/chosen: -261.5426
Logits/rejected: -0.3766
Logits/chosen: -0.3892

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.0652	2.67	100	1.7443	-5.2989	-4.9017	0.5100	-0.3972	-247.1705	-263.4201	-0.7895	-0.7659
0.0085	5.33	200	2.3642	-9.0530	-8.8642	0.4700	-0.1888	-255.0954	-270.9283	-0.4221	-0.4269
0.0	8.0	300	1.6817	-4.1578	-4.6561	0.5200	0.4983	-246.6793	-261.1379	-0.3697	-0.3824
0.0	10.67	400	1.6971	-4.2463	-4.7407	0.5300	0.4944	-246.8485	-261.3149	-0.3737	-0.3858
0.0	13.33	500	1.6857	-4.2849	-4.7855	0.5200	0.5006	-246.9381	-261.3922	-0.3756	-0.3880
0.0	16.0	600	1.7000	-4.3433	-4.8213	0.5300	0.4780	-247.0096	-261.5089	-0.3764	-0.3886
0.0	18.67	700	1.7016	-4.3573	-4.8617	0.5200	0.5044	-247.0905	-261.5370	-0.3772	-0.3894
0.0	21.33	800	1.6856	-4.3565	-4.8464	0.5100	0.4899	-247.0598	-261.5353	-0.3768	-0.3893
0.0	24.0	900	1.6949	-4.3569	-4.8528	0.5300	0.4959	-247.0727	-261.5360	-0.3774	-0.3892
0.0	26.67	1000	1.6900	-4.3601	-4.8463	0.5100	0.4862	-247.0597	-261.5426	-0.3766	-0.3892

Framework versions

PEFT 0.10.0
Transformers 4.39.1
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2