Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
✉️Corresponding Author
📝 Abstract
Reinforcement learning (RL) has proven highly effective in eliciting the reasoning capabilities of large language models (LLMs). Inspired by this success, recent studies have explored applying similar techniques to vision-language models (VLMs), aiming to enhance their reasoning performance.
However, directly transplanting RL methods from LLMs to VLMs is suboptimal, as the tasks faced by VLMs are inherently more complex. Specifically, VLMs must first accurately perceive and understand visual inputs before reasoning can be effectively performed.
To address this challenge, we propose a two-stage reinforcement learning framework designed to jointly enhance both the perceptual and reasoning capabilities of VLMs. To mitigate the vanishing advantage issue commonly observed in RL training, we first perform dataset-level sampling to selectively strengthen specific capabilities using distinct data sources.
During training:
- Stage 1 (Perception RL): Improves the model’s visual perception through coarse- and fine-grained visual understanding.
- Stage 2 (Reasoning RL): Targets the enhancement of reasoning abilities.
After the proposed two-stage reinforcement learning process, we obtain PeBR-R1, a vision-language model with significantly enhanced perceptual and reasoning capabilities.
Experimental results on seven benchmark datasets demonstrate the effectiveness of our approach and validate the superior performance of PeBR-R1 across diverse visual reasoning tasks.
🔗 Project Links
Citation
If you find PeBR-R1 useful for your research, please consider citing our work:
1@article{chen2025perception,
2 title={Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models},
3 author={Chen, Yan and Li, Long and Xi, Teng and Zeng, Long and Wang, Jingdong},
4 journal={arXiv preprint arXiv:2509.13031},
5 year={2025}
6}
