Views
No views yet
| Parameter | Value |
|---|---|
| Base model | ontocord/1.7b-MixtureVitae-300BT-v1-decontaminated |
| Dataset | ontocord/MV_Qwen-Magpie (1048 examples) |
| Method | SFT |
| Epochs | 2 |
| Learning rate | 5e-6 |
| LR scheduler | linear |
| Warmup ratio | 0.03 |
| Effective batch size | 128 |
| Per-device batch size | 16 |
| Max sequence length | 4096 |
| Precision | bf16 |
| Gradient checkpointing | true |
| Chat template | tulu3 |
| Packing | false |
| DeepSpeed | ZeRO Stage 2 |
| Optimizer | AdamW (beta1=0.9, beta2=0.999, eps=1e-8) |
| Weight decay | 0.0 |
| Seed | 42 |
| Nodes | 2 x 4 GPUs (H100) |