wandb: Run history:
wandb:               eval/loss █▃▂▂▂▁▂▁▁▁▁▁▁▁▁▂
wandb:            eval/runtime ▆▂▅▂▄▁█▂▃▁▃▂▂▅▃▄
wandb: eval/samples_per_second ▃▆▄▇▅█▁▇▆█▆▇▇▄▆▅
wandb:   eval/steps_per_second ▃▆▄▇▅█▁▇▆█▆▇▇▄▆▅
wandb:             train/epoch ▁▁▁▂▂▂▂▂▂▃▃▃▄▄▄▄▄▄▄▄▅▅▅▅▅▆▆▆▆▆▇▇▇▇▇▇▇███
wandb:       train/global_step ▁▁▁▁▂▂▂▂▃▃▃▃▃▃▃▄▄▄▄▄▅▅▅▅▅▆▆▆▆▆▆▇▇▇▇▇████
wandb:         train/grad_norm ▅▂▂▂█▂▂▁▁▁▂▁▁▁▂▁▁▂▁▂▁▁▂▂▁▁▁▁▂▂▁▂▂▂▂▂▂▂▁▂
wandb:     train/learning_rate ████▇▇▇▇▇▇▆▆▆▆▆▅▅▅▅▅▄▄▄▄▄▄▃▃▃▃▃▂▂▂▂▂▂▁▁▁
wandb:              train/loss █▃▃▂▁▂▂▂▁▃▂▂▁▃▂▂▁▃▂▂▁▃▂▂▂▃▂▁▃▂▁▁▃▂▂▁▃▂▂▁
wandb: 
wandb: Run summary:
wandb:                eval/loss 0.50501
wandb:             eval/runtime 49.8148
wandb:  eval/samples_per_second 6.745
wandb:    eval/steps_per_second 2.248
wandb:               total_flos 1.7024284869758976e+16
wandb:              train/epoch 1.45897
wandb:        train/global_step 480
wandb:          train/grad_norm 0.46409
wandb:      train/learning_rate 0.00014
wandb:               train/loss 0.2886
wandb:               train_loss 0.55963
wandb:            train_runtime 2282.1745
wandb: train_samples_per_second 5.766
wandb:   train_steps_per_second 0.721

training_arguments = SFTConfig(
    output_dir=new_model,
    run_name="fine_tune_ocr_correction",
    per_device_train_batch_size=2, #8
    per_device_eval_batch_size=3,
    gradient_accumulation_steps=4,
    optim="paged_adamw_32bit",
    num_train_epochs=5, 
    eval_strategy="steps",
    eval_steps=30,  
    save_steps=30,
    logging_steps=10,  
    warmup_steps=10,
    logging_strategy="steps",
    learning_rate= 2e-4, #5e-5,
    fp16=use_fp16, 
    bf16=use_bf16,  
    group_by_length=True,
    report_to="wandb",
    max_seq_length=1220,
    save_strategy="steps",
    dataset_text_field="text",
    load_best_model_at_end = True
)