learning_rate: 3e-05
train_batch_size: 8
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 1
mixed_precision_training: Native AMP

Training Loss	Epoch	Step	Validation Loss
3.6082	0.0757	200	3.4527
3.4472	0.1514	400	3.3252
3.3842	0.2271	600	3.2697
3.3915	0.3027	800	3.2255
3.3209	0.3784	1000	3.1980
3.3036	0.4541	1200	3.1748
3.2422	0.5298	1400	3.1566
3.2840	0.6055	1600	3.1415
3.2755	0.6812	1800	3.1303
3.2236	0.7569	2000	3.1217
3.1934	0.8325	2200	3.1143
3.2567	0.9082	2400	3.1107
3.2337	0.9839	2600	3.1097

UaKidDev/distilgpt2-instruct

distilgpt2-instruct