This is a 32B reasoning model trained from Qwen2.5-32B-Instruct with 17K data. The performance is on par with o1-preview model on both math and coding.
Please see our
blog post for more details.
17K verified correct responses from Qwen/QwQ-32B-Preview on coding, math. In addition, we add the science portion from the
Still-2 paper.
We perform supervised fine tuning on the data, with a batch size of 96.
We use Llama-Factory for training. On 8 H100, the training takes 19 hours with DeepSpeed Zero-3 Offload.
We would like to thanks the compute resources from
Lambda Lab and
AnyScale. We would like to thanks the academic feedback and support from the
Still-2 Team, and
Junyang Lin from the
Qwen Team.
Please considering citing our blog post if you found it useful for your research. Thank you!
1@misc{sky_t1_2025,
2 author = {NovaSky Team},
3 title = {Sky-T1: Fully open-source reasoning model with o1-preview performance in $450 budget},
4 howpublished = {https://novasky-ai.github.io/posts/sky-t1},
5 note = {Accessed: 2025-01-09},
6 year = {2025}
7}
