learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
3.3953	0.3384	200	2.9406
2.8055	0.6768	400	2.4826
2.4613	1.0152	600	2.2880
2.3053	1.3536	800	2.1368
2.1873	1.6920	1000	2.0303
2.1111	2.0305	1200	1.9459
2.0401	2.3689	1400	1.8865
1.9808	2.7073	1600	1.8239
1.9352	3.0457	1800	1.7836
1.8943	3.3841	2000	1.7307
1.8537	3.7225	2200	1.6958
1.818	4.0609	2400	1.6524
1.7887	4.3993	2600	1.6286
1.7576	4.7377	2800	1.5956
1.7308	5.0761	3000	1.5643
1.7099	5.4146	3200	1.5433
1.6858	5.7530	3400	1.5231
1.6614	6.0914	3600	1.4942
1.643	6.4298	3800	1.4843
1.6264	6.7682	4000	1.4670
1.611	7.1066	4200	1.4405
1.5892	7.4450	4400	1.4312
1.5784	7.7834	4600	1.4112
1.5632	8.1218	4800	1.3984
1.5562	8.4602	5000	1.3866
1.5295	8.7986	5200	1.3731
1.5294	9.1371	5400	1.3648
1.5089	9.4755	5600	1.3476
1.4992	9.8139	5800	1.3422
1.491	10.1523	6000	1.3284
1.4795	10.4907	6200	1.3250
1.4691	10.8291	6400	1.3100
1.4658	11.1675	6600	1.3045
1.4559	11.5059	6800	1.2981
1.4403	11.8443	7000	1.2853
1.4333	12.1827	7200	1.2739
1.4336	12.5212	7400	1.2707
1.4256	12.8596	7600	1.2576
1.4286	13.1980	7800	1.2573
1.4062	13.5364	8000	1.2492
1.3999	13.8748	8200	1.2432
1.3961	14.2132	8400	1.2377
1.3955	14.5516	8600	1.2293
1.3889	14.8900	8800	1.2240
1.3823	15.2284	9000	1.2162
1.3682	15.5668	9200	1.2242
1.3738	15.9052	9400	1.2117
1.366	16.2437	9600	1.2141
1.3612	16.5821	9800	1.2045
1.3663	16.9205	10000	1.1962
1.3583	17.2589	10200	1.2003
1.3454	17.5973	10400	1.1925
1.351	17.9357	10600	1.1902
1.3453	18.2741	10800	1.1859
1.3396	18.6125	11000	1.1789
1.3386	18.9509	11200	1.1823
1.3418	19.2893	11400	1.1769
1.3303	19.6277	11600	1.1704
1.329	19.9662	11800	1.1686
1.3202	20.3046	12000	1.1718
1.3241	20.6430	12200	1.1659
1.3246	20.9814	12400	1.1658
1.32	21.3198	12600	1.1670
1.3208	21.6582	12800	1.1610
1.3169	21.9966	13000	1.1672
1.3147	22.3350	13200	1.1569
1.3164	22.6734	13400	1.1664
1.319	23.0118	13600	1.1592
1.3108	23.3503	13800	1.1611
1.3129	23.6887	14000	1.1562
1.3121	24.0271	14200	1.1593
1.3077	24.3655	14400	1.1552
1.3144	24.7039	14600	1.1533

Framework versions

Transformers 4.57.1
Pytorch 2.8.0+cu128
Datasets 4.0.0
Tokenizers 0.22.0

JiangCilter/stage1_mlm___tester

stage1_mlm___tester

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

JiangCilter/
stage1_mlm___tester