learning_rate=4e-6,
lr_scheduler_type="linear",
beta=0.1,
per_device_train_batch_size=4,
per_device_eval_batch_size=4,
gradient_accumulation_steps=8,
optim="paged_adamw_8bit",
num_train_epochs=1,| Metric | Value |
|---|---|
| Avg. | 29.45 |
| IFEval (0-Shot) | 63.31 |
| BBH (3-Shot) | 40.12 |
| MATH Lvl 5 (4-Shot) | 16.69 |
| GPQA (0-shot) | 12.42 |
| MuSR (0-shot) | 9.80 |
| MMLU-PRO (5-shot) | 34.39 |