This repository contains
CoMa-3B, a multimodal embedding model presented in the paper
Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding.
CoMa (Compression then Matching) is an efficient pre-training paradigm designed to transform Multimodal Large Language Models (MLLMs) into competitive embedding models. It introduces a compressed pre-training phase that serves as a warm-up stage for contrastive learning, allowing the model to comprehensively preserve semantic content while emphasizing discriminative features for downstream tasks. CoMa achieves state-of-the-art results among MLLMs of comparable size on the MMEB benchmark.
1@article{li2025compressing,
2 title={Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding},
3 author={Li, Da and Luo, Yuxiao and Bi, Keping and Guo, Jiafeng and Yuan, Wei and Yang, Biao and Wang, Yan and Yang, Fan and Gao, Tingting and Zhou, Guorui},
4 journal={arXiv preprint arXiv:2511.08480},
5 year={2025}
6}
