You can deploy this model and start earning money today!

TARARARAK/
HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-32_64_3

Name: HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-32_64_3
Availability: InStock

Views

No views yet

HGU_rulebook-Llama3.2-Bllossom-5B_fine-tuning-QLoRA-32_64_3

This model is a fine-tuned version of Bllossom/llama-3.2-Korean-Bllossom-AICA-5B on an unknown dataset. It achieves the following results on the evaluation set:

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 16
optimizer: Use adamw_bnb_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine_with_restarts
lr_scheduler_warmup_ratio: 0.1
training_steps: 942
mixed_precision_training: Native AMP

Training Loss	Epoch	Step	Validation Loss
12.459	0.2991	47	11.8428
8.9527	0.5982	94	8.3608
6.8438	0.8974	141	6.6652
6.1272	1.1965	188	6.0176
5.858	1.4956	235	5.8316
5.7707	1.7947	282	5.7618
5.7435	2.0939	329	5.7331
5.7215	2.3930	376	5.7194
5.7118	2.6921	423	5.7114
5.7047	2.9912	470	5.7063
5.703	3.2904	517	5.7027
5.6973	3.5895	564	5.7001
5.6959	3.8886	611	5.6985
5.6921	4.1877	658	5.6972
5.6968	4.4869	705	5.6965
5.6954	4.7860	752	5.6959
5.6958	5.0851	799	5.6955
5.6937	5.3842	846	5.6953
5.6915	5.6834	893	5.6952
5.6951	5.9825	940	5.6952