learning_rate: 0.0006
train_batch_size: 32
eval_batch_size: 32
seed: 41
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 2000
num_epochs: 20.0
mixed_precision_training: Native AMP

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	971	4.4042	0.3295
1.2033	2.0	1942	4.1879	0.3475
0.9928	3.0	2913	4.0195	0.3588
0.938	4.0	3884	3.9204	0.3674
0.895	5.0	4855	3.8516	0.3726
0.8675	6.0	5826	3.8064	0.3757
0.8468	7.0	6797	3.7914	0.3783
0.8258	8.0	7768	3.7569	0.3812
0.8125	9.0	8739	3.7413	0.3831
0.7971	10.0	9710	3.7261	0.3846
0.7838	11.0	10681	3.7196	0.3855
0.7706	12.0	11652	3.7084	0.3872
0.7576	13.0	12623	3.7034	0.3881
0.7458	14.0	13594	3.7016	0.3890
0.733	15.0	14565	3.7040	0.3895
0.721	16.0	15536	3.7076	0.3899
0.7109	17.0	16507	3.7013	0.3908
0.6999	18.0	17478	3.7044	0.3913
0.6897	19.0	18449	3.7064	0.3916
0.6796	20.0	19420	3.7071	0.3920

xiulinyang/GPT2_10M_50000

10M_50000_41