You can deploy this model and start earning money today!

TARARARAK/
HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-32_128_2

Name: HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-32_128_2
Availability: InStock

Views

No views yet

HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-32_128_2

This model is a fine-tuned version of Bllossom/llama-3.2-Korean-Bllossom-AICA-5B on an unknown dataset. It achieves the following results on the evaluation set:

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 16
optimizer: Use adamw_bnb_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine_with_restarts
lr_scheduler_warmup_ratio: 0.1
training_steps: 628
mixed_precision_training: Native AMP

Training Loss	Epoch	Step	Validation Loss
8.0386	0.1973	31	6.9446
5.7807	0.3946	62	5.7341
5.6962	0.5919	93	5.6926
5.6881	0.7892	124	5.6864
5.6812	0.9865	155	5.6828
5.678	1.1838	186	5.6816
5.6779	1.3811	217	5.6804
5.6752	1.5784	248	5.6796
5.6755	1.7757	279	5.6794
5.6795	1.9730	310	5.6787
5.6751	2.1702	341	5.6784
5.6748	2.3675	372	5.6783
5.672	2.5648	403	5.6780
5.6731	2.7621	434	5.6777
5.6714	2.9594	465	5.6776
5.6734	3.1567	496	5.6779
5.6707	3.3540	527	5.6778
5.6711	3.5513	558	5.6778
5.671	3.7486	589	5.6778
5.6719	3.9459	620	5.6778