LaME augments Qwen-VL with learnable [REASON] tokens and a latent decoder supervision, jointly optimizing generation and embedding through an information bottleneck. It produces both discriminative and generative multimodal embeddings for text, images, videos, and visual documents.

Without bells and whistles, LaME achieves state-of-the-art multimodal retrieval performance on MMEB-v2 (image / video / visual-document / full aggregate) and MRMR.

Backbone: Qwen2-VL-7B-Instruct
Latent Decoder: Qwen3-0.6B
Reason Tokens: 8 learnable [REASON] tokens
Projection Dim: 3584
Training Stage: 2 (joint contrastive + decoder-supervised)

Usage

See the LaME repository for inference and evaluation examples.

python

from transformers import AutoModel, AutoProcessor

model = AutoModel.from_pretrained("leafyseay/LaME-7B", trust_remote_code=True, torch_dtype="bfloat16").cuda()
processor = AutoProcessor.from_pretrained("leafyseay/LaME-7B", trust_remote_code=True)

Citation

bibtex

@article{wu2026lame,
  title   = {LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck},
  author  = {Wu, Peixi and Yang, Biao and Ma, Feipeng and Chai, Bosong and Lin, Bo and Yuan, Wei and Yang, Fan and Gao, Tingting and Li, Hebei and Sun, Xiaoyan},
  journal = {arXiv preprint arXiv:2606.13061},
  year    = {2026}
}