This model was presented in the paper Reasoning in the Dark: Interleaved Vision-Text Reasoning in Latent Space.

Interleaved Vision-Text Latent Reasoning (IVT-LR) is the first VLM framework that unifies textual and visual representations in the latent space and implements multimodal latent reasoning. Specifically, IVT-LR represents each reasoning step by combining two implicit parts: latent text and latent vision. We further introduce a progressive multi-stage training strategy to enable MLLMs to perform the above multimodal latent reasoning steps.

This repository provides pretrained models for Qwen2-VL on M3CoT and Chameleon on ScienceQA.

To see detailed usage, including inference code and scripts for training, please refer to the GitHub repository.

You can download the models directly from Hugging Face using huggingface_hub: