You can deploy this model and start earning money today!

TARARARAK/
HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-16_64_3

Name: HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-16_64_3
Availability: InStock

Views

No views yet

HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-16_64_3

This model is a fine-tuned version of Bllossom/llama-3.2-Korean-Bllossom-AICA-5B on an unknown dataset. It achieves the following results on the evaluation set:

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 16
optimizer: Use adamw_bnb_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine_with_restarts
lr_scheduler_warmup_ratio: 0.1
training_steps: 942
mixed_precision_training: Native AMP

Training Loss	Epoch	Step	Validation Loss
7.7538	0.2991	47	6.4144
5.7228	0.5982	94	5.7066
5.6895	0.8974	141	5.6889
5.6821	1.1965	188	5.6843
5.6823	1.4956	235	5.6830
5.683	1.7947	282	5.6813
5.6833	2.0939	329	5.6805
5.678	2.3930	376	5.6806
5.6776	2.6921	423	5.6796
5.6752	2.9912	470	5.6791
5.6757	3.2904	517	5.6791
5.6722	3.5895	564	5.6788
5.6735	3.8886	611	5.6786
5.6683	4.1877	658	5.6788
5.6744	4.4869	705	5.6786
5.6738	4.7860	752	5.6785
5.6734	5.0851	799	5.6785
5.6709	5.3842	846	5.6786
5.6698	5.6834	893	5.6786
5.6734	5.9825	940	5.6786