This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mzhaoshuai/llama3-ultrafeedback-bertscore-bart-large-mnli dataset.
It achieves the following results on the evaluation set:
Loss: 1.3795
Rewards/chosen: -9.7179
Rewards/rejected: -15.1619
Rewards/accuracies: 0.8770
Rewards/margins: 5.4440
Logps/rejected: -1.5162
Logps/chosen: -0.9718
Logits/rejected: -1.3590
Logits/chosen: -1.2077
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
learning_rate: 1e-06
train_batch_size: 2
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 16
total_train_batch_size: 128
total_eval_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
