Views
No views yet
exp2199b_redo3 experiment, which fine-tunes Qwen2.5-7B-Instruct on the OpenThoughts3 dataset.| Parameter | Value |
|---|---|
| Epochs | 5 |
| Batch Size | 512 |
| Learning Rate | 8e-5 |
| Max Sequence Length | 16384 |
| LR Schedule | Cosine |
| Warmup | 10% |
| Decay | 0.9 |
| Weight Decay | 0.0 |
| Beta1 | 0.9 |
| Beta2 | 0.999 |
| Hardware | TPU v4-512 |