You can deploy this model and start earning money today!

mifeng09/
llama_lora_20k_alpaca_begining

Name: llama_lora_20k_alpaca_begining
Availability: InStock

Views

No views yet

llama_lora_20k_alpaca_begining

This model is a fine-tuned version of mifeng09/my_final_llama_model_v2_add_wiki_fix_resume on an unknown dataset. It achieves the following results on the evaluation set:

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 4
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
lr_scheduler_warmup_steps: 101
num_epochs: 3

Training Loss	Epoch	Step	Validation Loss
4.2374	0.1484	50	4.0144
4.3818	0.2967	100	3.9270
3.944	0.4451	150	3.8766
3.3377	0.5935	200	3.8441
3.3489	0.7418	250	3.8238
3.5876	0.8902	300	3.8094
3.7506	1.0386	350	3.7967
3.9164	1.1869	400	3.7891
4.4806	1.3353	450	3.7829
4.078	1.4837	500	3.7769
3.8764	1.6320	550	3.7716
3.7083	1.7804	600	3.7681
3.5106	1.9288	650	3.7660
3.8436	2.0772	700	3.7631
3.8274	2.2255	750	3.7621
4.0989	2.3739	800	3.7603
3.7672	2.5223	850	3.7594
3.7825	2.6706	900	3.7592
3.8184	2.8190	950	3.7591
4.0853	2.9674	1000	3.7591