learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 32
optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
num_epochs: 5

Training Loss	Epoch	Step	Validation Loss	Bleu	Exact Match
56.4754	0.2353	500	20.0328	2.1011	0.0
22.0279	0.4706	1000	7.7486	2.4811	0.0
9.4210	0.7059	1500	3.6371	0.6107	0.0
5.1025	0.9412	2000	1.9162	0.3536	0.0
4.2424	1.1765	2500	1.7520	0.0741	0.0
3.8246	1.4118	3000	1.6597	0.2389	0.0
3.4564	1.6471	3500	1.2625	0.9807	0.0
2.1192	1.8824	4000	0.2615	17.8148	0.0
1.1416	2.1176	4500	0.2232	26.2052	0.0008
0.8033	2.3529	5000	0.2074	41.9012	0.0018
0.6566	2.5882	5500	0.1916	46.3403	0.0026
0.5929	2.8235	6000	0.1769	50.3816	0.004
0.5420	3.0588	6500	0.1628	55.1438	0.0064
0.4957	3.2941	7000	0.1504	59.1973	0.0089
0.4878	3.5294	7500	0.1396	62.7263	0.0111
0.4614	3.7647	8000	0.1317	65.2962	0.0127
0.4208	4.0	8500	0.1266	67.1441	0.0162
0.4115	4.2353	9000	0.1228	68.7721	0.0174
0.4289	4.4706	9500	0.1210	69.3114	0.0185
0.3984	4.7059	10000	0.1197	69.7885	0.0189
0.4047	4.9412	10500	0.1194	69.8427	0.019
0.3908	5.0	10625	0.1194	69.8553	0.0192

SPEAK-PP/mt5-small-si-spelling-correction-dataset-synthetic-100k

mt5-small-si-spelling-correction-dataset-synthetic-100k