We focus on the mathematical reasoning distillation task and propose a novel method LoRID, which draws inspiration from the human beings teaching and learning pattern.
We introduce knowledge during data augmentation and propose multi-LoRA interaction during model distillation, which improves the student’s reasoning abilities.
Experimental results show that with the interaction between System 1 and System 2, LoRID outperforms previous state-of-the-art approaches and can be easily and effectively integrated into any Chain-of-Thought distillation method.

Citation

If this work is helpful, please kindly cite as:

bibtex

@misc{li2025largemodelsteachstudent,
      title={Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction}, 
      author={Xinhe Li and Jiajun Liu and Peng Wang},
      year={2025},
      eprint={2508.13037},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.13037}, 
}

LoRID-Math/MATH-Mistral-7B-KG

LoRID: A Reasoning Distillation Method via Multi-LoRA Interaction

Abstract

Key Contributions

Citation

LoRID-Math/
MATH-Mistral-7B-KG