learning_rate: 5e-06
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 16
gradient_accumulation_steps: 2
total_train_batch_size: 32
total_eval_batch_size: 16
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3.0

Training Loss	Epoch	Step	Validation Loss
0.9373	0.1778	100	0.9672
0.9671	0.3556	200	0.9639
0.9584	0.5333	300	0.9629
0.957	0.7111	400	0.9597
0.9477	0.8889	500	0.9587
0.8552	1.0667	600	0.9710
0.7944	1.2444	700	0.9722
0.7359	1.4222	800	0.9709
0.8494	1.6	900	0.9662
0.8163	1.7778	1000	0.9663
0.8041	1.9556	1100	0.9639
0.6291	2.1333	1200	0.9990
0.6122	2.3111	1300	1.0004
0.6718	2.4889	1400	1.0003
0.6712	2.6667	1500	1.0002
0.6397	2.8444	1600	0.9996

w3en2g/qa-Qwen2.5-32B-Base

Qwen2.5-32B-Base