SAIL-VL2

🔥 Updates

• 2025.09.18 🌟 SAIL-VL2 Technical Report is now available at arxiv.

🌟 Highlights

• SAIL-VL2 is powerful, efficient, and achieves top results under 2B parameters.
• SAIL-VL2-Thinking boosts complex reasoning, matching larger models.
• SAIL-VL2 excels in fine-grained visual tasks beyond similar-scale models.

Model Architecture:

🎬 Quick Start

👀 Introduction

• SAIL-VL2 is powerful yet efficient: With training on 776B tokens, SAIL-VL2 has verified its effectiveness across 106 datasets, achieving state-of-the-art results on a broad spectrum of influential benchmarks under the 2B-parameter scale. Remarkably, even without specialized prompting, the base SAIL-VL2 model delivers highly competitive performance on challenging reasoning benchmarks such as MMMU and MathVista, demonstrating strong out-of-the-box capabilities.
• SAIL-VL2 as a deep thinker: Many real-world tasks demand sophisticated reasoning and multi-step thought processes, which remain challenging for standard LVMs. To address this, we develop SAIL-VL2-Thinking, a specialized variant trained with advanced Chain-of-Thought (CoT) and reinforcement learning (RL) strategies. This design substantially improves performance on complex reasoning benchmarks, often matching or even surpassing models with far larger parameter scales, thereby setting a new standard for efficient architectures in high-level reasoning.
• SAIL-VL2 perceives with clarity: Fine-grained visual understanding is a critical challenge for multimodal models. SAIL-VL2 delivers high-fidelity perception in tasks such as OCR, high-resolution document layout analysis, and complex chart interpretation, achieving detailed visual grounding beyond models of similar scale.

Overview of the SAIL-VL2 framework. The architecture is composed of a vision encoder that aligns visual inputs into the representation space of the LLM. A lightweight adapter further transforms visual embeddings into tokenized representations, which are jointly processed with linguistic embeddings for multimodal reasoning and prediction. SAIL-VL2 accommodates multiple LLM backbones, ensuring flexibility and scalability across model configurations.

📚 Training Strategy

🌟 Data construction

Data construction pipeline for SAIL-VL2 training. High-quality multimodal corpora are construted by curating and filtering open-source datasets, and generating synthetic data, with both components systematically organized to meet the requirements of different training stages.

🌟 Pre-Train

• Basic Multimodal Pre-Training: develops SAIL-VL2’s multimodal alignment via SAIL-ViT, LLM and a random MLP adapter, using 64M samples, AdaLRS and 2048 batch size.
• Multi-task Pre-Training: strengthens SAIL-VL2’s visual and instruction-following abilities, unfreezes all params, adds instruction-tuning data, uses 180M samples, and skips AdaLRS.

Scaling curves of SAIL-VL2-2B during the multi-task pre-training stage. Results are reported on overall benchmarks, natural-scene VQA datasets, and OCR VQA tasks. ’BMK Score’ denotes the average benchmark score.

🌟 Post-Train

• Basic Supervised Fine-Tuning: 4 phases; Model Soup merges homogeneous models.
• LongCoT Supervised Fine-tuning: enhances the model’s step-by-step reasoning capabilities for complex problems.
• RL with Verifiable Rewards: refines the model by optimizing it against a reward system focused on two primary objectives: the correctness of the final answer and adherence to the specified output format
• Think-Fusion Supervised Fine-tuning: enhances the model’s reasoning capabilities while maintaining its broad general understanding.
• RL with a Mixed Reward System: enhances the model’s reasoning capabilities through a RL stage

📈 Experimental Results

🌟 Performance of 2B series

Overall comparison of the SAIL-VL2 series and existing open-source MLLMs (<4B).

🌟 Performance of 8B series

Overall comparison of the SAIL-VL2 series with existing open-source 8B MLLMs and closed-source models.

🌟 Performance of Thinking-mode models

Evaluation results on OpenCompass multimodal reasoning benchmarks.

🙏 Acknowledge

✒️ Citation

@misc{yin2025sailvl2technicalreport,
  title={SAIL-VL2 Technical Report}, 
  author={Weijie Yin and Yongjie Ye and Fangxun Shu and Yue Liao and Zijian Kang and Hongyuan Dong and Haiyang Yu and Dingkang Yang and Jiacong Wang and Han Wang and Wenzhuo Liu and Xiao Liang and Shuicheng Yan and Chao Feng},
  year={2025},
  eprint={2509.14033},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2509.14033}, 
}

@article{dong2025scalable,
  title={Scalable vision language model training via high quality data curation},
  author={Dong, Hongyuan and Kang, Zijian and Yin, Weijie and Liang, Xiao and Feng, Chao and Ran, Jiao},
  journal={arXiv preprint arXiv:2501.05952},
  year={2025}
}

📜 License

📧Contact