Maestro-R1-Llama-8B is a powerful language model fine-tuned from DeepSeek-R1-Distill-Llama-8B, a distilled model based on the Llama-3 architecture. DeepSeek-R1-Distill-Llama-8B itself is derived from the Llama-3 architecture, with a distillation process from DeepSeek-R1, utilizing a large corpus of diverse data. This distillation enables the model to retain strong reasoning capabilities while maintaining a smaller parameter count.
Maestro-R1-Llama-8B builds on this foundation, further enhancing its performance through fine-tuning on the ServiceNow-AI/R1-Distill-SFT dataset. This fine-tuning step sharpens the model's ability to handle specialized tasks and improves its reasoning, problem-solving, and code generation capabilities. The combination of the distilled base model and domain-specific fine-tuning makes Maestro-R1-Llama-8B an efficient and robust model, excelling across a wide range of language tasks.
DeepSeek-R1 Paper Link: https://arxiv.org/abs/2501.12948