You can deploy this model and start earning money today!

qfq/
Llama-3.1-8B-Instruct-EI1-2ep-sft-warmupbs

Name: Llama-3.1-8B-Instruct-EI1-2ep-sft-warmupbs
Availability: InStock

generated_from_trainer

meta-llama/Llama-3.1-8B-Instruct

finetune

llama3.1

Views

No views yet

Llama-3.1-8B-Instruct-EI1-2ep-sft-warmupbs

This model is a fine-tuned version of meta-llama/Llama-3.1-8B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.6092

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 6e-06
train_batch_size: 1
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 16
total_train_batch_size: 16
total_eval_batch_size: 128
optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.1403	100	0.6723
No log	0.2805	200	0.6387
No log	0.4208	300	0.6302
No log	0.5610	400	0.6238
0.6538	0.7013	500	0.6162
0.6538	0.8415	600	0.6104
0.6538	0.9818	700	0.6053
0.6538	1.1220	800	0.6186
0.6538	1.2623	900	0.6180
0.4905	1.4025	1000	0.6138
0.4905	1.5428	1100	0.6119
0.4905	1.6830	1200	0.6103
0.4905	1.8233	1300	0.6093
0.4905	1.9635	1400	0.6092

Framework versions

Transformers 4.43.4
Pytorch 2.4.0+cu121
Datasets 3.0.1
Tokenizers 0.19.1

qfq/Llama-3.1-8B-Instruct-EI1-2ep-sft-warmupbs

Llama-3.1-8B-Instruct-EI1-2ep-sft-warmupbs

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

qfq/
Llama-3.1-8B-Instruct-EI1-2ep-sft-warmupbs