This repository contains a model distilled using
Align-TI, a novel knowledge distillation (KD) framework introduced in the paper
Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactions.
Align-TI is designed to compress Multimodal Large Language Models (MLLMs) by focusing on dynamic token interactions rather than just static next-token alignment. It introduces two primary components:
The framework achieves state-of-the-art performance for parameter-efficient MLLMs, with the 2B version even outperforming significantly larger models like LLaVA-1.5-7B.
1@article{chen2026alignti,
2 title={Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactions},
3 author={Lin Chen and Xiaoke Zhao and Kun Ding and Weiwei Feng and Changtao Miao and Zili Wang and Wenxuan Guo and Ying Wang and Kaiyuan Zheng and Bo Zhang and Zhe Li and Shiming Xiang},
4 journal={arXiv preprint arXiv:2602.09483},
5 year={2026},
6}
