learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
training_steps: 500
mixed_precision_training: Native AMP

Training Loss	Epoch	Step	Validation Loss
12.9134	0.0003	20	12.2793
9.4671	0.0005	40	8.7011
4.7451	0.0008	60	4.3410
3.8301	0.0011	80	3.7443
3.5732	0.0014	100	3.6002
3.6253	0.0016	120	3.5792
3.5599	0.0019	140	3.5657
3.6170	0.0022	160	3.5588
3.5905	0.0025	180	3.5539
3.6341	0.0027	200	3.5503
3.4940	0.0030	220	3.5481
3.5236	0.0033	240	3.5462
3.5430	0.0035	260	3.5500
3.5657	0.0038	280	3.5444
3.5832	0.0041	300	3.5426
3.5796	0.0044	320	3.5418
3.4968	0.0046	340	3.5411
3.5693	0.0049	360	3.5405
3.5740	0.0052	380	3.5397
3.5217	0.0055	400	3.5394
3.5725	0.0057	420	3.5391
3.5787	0.0060	440	3.5388
3.5265	0.0063	460	3.5387
3.5717	0.0065	480	3.5383
3.5212	0.0068	500	3.5383

ueihieu/llama2-7b-lora-genwiki

llama2-7b-lora-genwiki