learning_rate: 0.0002
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 32
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.03
num_epochs: 5

Training Loss	Epoch	Step	Validation Loss
2.5892	0.0674	100	1.9698
2.2573	0.1347	200	2.0053
2.1211	0.2021	300	1.9714
2.1551	0.2695	400	1.9574
2.181	0.3368	500	1.9433
2.1707	0.4042	600	1.9536
2.1463	0.4715	700	1.9302
2.1451	0.5389	800	1.9041
2.1099	0.6063	900	1.9010
2.1178	0.6736	1000	1.9308
2.1292	0.7410	1100	1.9336
2.1285	0.8084	1200	1.8931
2.0634	0.8757	1300	1.9067
2.1118	0.9431	1400	1.9244
2.0807	1.0101	1500	1.8401
2.0402	1.0775	1600	1.8851
2.1129	1.1448	1700	1.8613
2.0707	1.2122	1800	1.9472

Ratul-Chakraborty/BERT

BERT