You can deploy this model and start earning money today!

aaaalongaa/
llama3-8b-dpo-full-lr4e-7

Name: llama3-8b-dpo-full-lr4e-7
Rating: 2.5 (6 reviews)

generated_from_trainer

princeton-nlp/Llama-3-Base-8B-SFT

finetune

Views

No views yet

llama3-8b-dpo-full-lr4e-7

This model is a fine-tuned version of princeton-nlp/Llama-3-Base-8B-SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.4982
Rewards/chosen: 0.0514
Rewards/rejected: -1.0809
Rewards/accuracies: 0.7679
Rewards/margins: 1.1323
Logps/rejected: -292.1989
Logps/chosen: -302.1581
Logits/rejected: -1.0585
Logits/chosen: -1.0327

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 4e-07
train_batch_size: 4
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 2
total_train_batch_size: 32
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6632	0.0523	100	0.6628	0.2140	0.1331	0.6746	0.0809	-280.0588	-300.5320	-1.0266	-0.9965
0.5726	0.1047	200	0.5856	0.4133	-0.0580	0.7242	0.4713	-281.9698	-298.5389	-1.0532	-1.0245
0.555	0.1570	300	0.5530	0.2693	-0.4045	0.75	0.6738	-285.4353	-299.9796	-1.0721	-1.0447
0.5235	0.2094	400	0.5374	-0.5619	-1.3822	0.7659	0.8203	-295.2119	-308.2914	-1.0998	-1.0712
0.4803	0.2617	500	0.5274	-0.2470	-1.1343	0.7639	0.8873	-292.7331	-305.1427	-1.0561	-1.0298
0.5554	0.3141	600	0.5155	-0.0887	-1.0164	0.7639	0.9277	-291.5540	-303.5598	-1.0275	-1.0029
0.5353	0.3664	700	0.5128	0.3003	-0.6782	0.7718	0.9784	-288.1716	-299.6696	-1.0423	-1.0168
0.5653	0.4187	800	0.5093	0.1878	-0.7730	0.7758	0.9608	-289.1198	-300.7938	-1.0417	-1.0174
0.4951	0.4711	900	0.5036	-0.1484	-1.1678	0.7738	1.0193	-293.0678	-304.1566	-1.0381	-1.0143
0.5146	0.5234	1000	0.5027	-0.4460	-1.4969	0.7599	1.0509	-296.3586	-307.1322	-1.0642	-1.0380
0.4819	0.5758	1100	0.5027	0.2699	-0.7994	0.7639	1.0693	-289.3842	-299.9731	-1.0647	-1.0385
0.4805	0.6281	1200	0.5031	-0.2315	-1.2976	0.7698	1.0661	-294.3664	-304.9876	-1.0559	-1.0298
0.4771	0.6805	1300	0.5017	0.1269	-0.9837	0.7698	1.1107	-291.2273	-301.4029	-1.0511	-1.0256
0.4959	0.7328	1400	0.5018	0.1567	-0.9544	0.7718	1.1111	-290.9340	-301.1049	-1.0342	-1.0103
0.4622	0.7851	1500	0.4987	0.0275	-1.0933	0.7738	1.1208	-292.3228	-302.3973	-1.0290	-1.0053
0.4938	0.8375	1600	0.4987	0.1361	-0.9818	0.7639	1.1178	-291.2077	-301.3117	-1.0463	-1.0214
0.4702	0.8898	1700	0.4974	0.0840	-1.0426	0.7619	1.1265	-291.8157	-301.8326	-1.0570	-1.0313
0.4832	0.9422	1800	0.4976	0.0445	-1.0914	0.7659	1.1359	-292.3042	-302.2274	-1.0515	-1.0262
0.4862	0.9945	1900	0.4982	0.0514	-1.0809	0.7679	1.1323	-292.1989	-302.1581	-1.0585	-1.0327

Framework versions

Transformers 4.43.1
Pytorch 2.1.2+cu121
Datasets 2.18.0
Tokenizers 0.19.1

aaaalongaa/llama3-8b-dpo-full-lr4e-7

llama3-8b-dpo-full-lr4e-7

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

aaaalongaa/
llama3-8b-dpo-full-lr4e-7