| Attribute | Value |
|---|---|
| Base Model | Qwen3-0.6B |
| Teacher Model | SOD-GRPO_teacher-4B |
| Training Pipeline | Cold-Start SFT → SOD (Step-wise On-policy Distillation) |
| Parameters | 0.6B |
| Model | Description |
|---|---|
| SOD-0.6B | SOD-distilled 0.6B student (this model) |
| SOD-1.7B | SOD-distilled 1.7B student |
| SOD-GRPO_teacher-4B | GRPO-trained 4B teacher model |
| Method | AIME 2024 | AIME 2025 | GPQA-Diamond | LiveCodeBench-v6 | Average |
|---|---|---|---|---|---|
| Vanilla | 7.71 | 12.81 | 13.24 | 14.89 | 12.16 |
| SFT | 5.67 | 5.42 | 15.20 | 9.61 | 8.97 |
| GRPO | 4.06 | 4.90 | 20.38 | 15.95 | 11.32 |
| OPD | 16.82 | 22.95 | 17.76 | 22.65 | 20.04 |
| OPSD_gt | 12.63 | 17.04 | 17.32 | 16.73 | 15.93 |
| OPSD_hint | 9.77 | 14.12 | 15.98 | 12.65 | 13.13 |
| SOD (This Model) | 20.84 | 26.13 | 22.19 | 27.72 | 24.22 |
| Method | AIME 2024 | AIME 2025 | GPQA-Diamond | LiveCodeBench-v6 | Average |
|---|---|---|---|---|---|
| GRPO | 67.60 | 60.42 | 55.19 | 63.13 | 61.59 |
1@article{zhong2026sod,
2 title={SOD: Step-wise On-policy Distillation for Small Language Model Agents},
3 author={Zhong, Qiyong and Zheng, Mao and Song, Mingyang and Lin, Xin and Sun, Jie and Jiang, Houcheng and Wang, Xiang and Fang, Junfeng},
4 journal={arXiv preprint arXiv:2605.07725},
5 year={2026}
6}