You can deploy this model and start earning money today!

Angelectronic/
llama3-chat_50000_500

Name: llama3-chat_50000_500
Availability: InStock

generated_from_trainer

unsloth/llama-3-8b-Instruct-bnb-4bit

adapter

apache-2.0

Views

No views yet

llama3-chat_50000_500

This model is a fine-tuned version of unsloth/llama-3-8b-Instruct-bnb-4bit on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.7378

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 16
eval_batch_size: 16
seed: 3407
gradient_accumulation_steps: 4
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 5
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
1.7748	0.128	100	1.4928
1.5097	0.256	200	1.4743
1.5122	0.384	300	1.4623
1.4924	0.512	400	1.4609
1.485	0.64	500	1.4526
1.4779	0.768	600	1.4511
1.4728	0.896	700	1.4446
1.4476	1.024	800	1.4512
1.3725	1.152	900	1.4558
1.3747	1.28	1000	1.4560
1.3735	1.408	1100	1.4548
1.3717	1.536	1200	1.4499
1.3694	1.6640	1300	1.4526
1.3698	1.792	1400	1.4542
1.3701	1.92	1500	1.4512
1.3004	2.048	1600	1.4977
1.1904	2.176	1700	1.5075
1.1977	2.304	1800	1.5041
1.1888	2.432	1900	1.5094
1.1885	2.56	2000	1.5024
1.1989	2.6880	2100	1.5039
1.1905	2.816	2200	1.5046
1.1914	2.944	2300	1.5077
1.0764	3.072	2400	1.6027
0.9757	3.2	2500	1.6227
0.9768	3.328	2600	1.6228
0.9795	3.456	2700	1.6225
0.9775	3.584	2800	1.6190
0.9781	3.7120	2900	1.6164
0.981	3.84	3000	1.6199
0.9812	3.968	3100	1.6254
0.8731	4.096	3200	1.7307
0.8376	4.224	3300	1.7343
0.8352	4.352	3400	1.7398
0.8429	4.48	3500	1.7357
0.8431	4.608	3600	1.7386
0.8383	4.736	3700	1.7380
0.8375	4.864	3800	1.7376
0.842	4.992	3900	1.7378

Framework versions

PEFT 0.10.0
Transformers 4.40.2
Pytorch 2.3.0
Datasets 2.19.1
Tokenizers 0.19.1

Angelectronic/llama3-chat_50000_500

llama3-chat_50000_500

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Angelectronic/
llama3-chat_50000_500