MedCLIPSeg is a vision–language framework for medical image segmentation built on top of CLIP. It adapts CLIP for robust, data-efficient, and uncertainty-aware segmentation through probabilistic cross-modal attention and bidirectional interaction between image and text tokens.
The released checkpoints correspond exactly to the experiments reported in our paper and are provided for evaluation and reproducibility purposes only.
Download the checkpoints here, then create a directory named outputs_medclipseg at the root of the project and place the downloaded checkpoint folders inside it so that the directory structure matches the following layout:
The underlying vision–language models used in this repository were introduced in prior work. We gratefully acknowledge the original authors:
PubMedCLIP
Eslami, Sedigheh, Gerard De Melo, and Christoph Meinel. "Does clip benefit visual question answering in the medical domain as much as it does in the general domain?." arXiv preprint arXiv:2112.13906 (2021).
UniMedCLIP
Khattak, Muhammad Uzair, et al. "Unimed-clip: Towards a unified image-text pretraining paradigm for diverse medical imaging modalities." arXiv preprint arXiv:2412.10372 (2024).
For completeness and reproducibility, this repository also includes the original pretrained checkpoints of these foundation models under the checkpoints/ directory, exactly as released by their respective authors.
All MedCLIPSeg checkpoints are adaptations built on top of these pretrained models and are released strictly for research and non-commercial use, in accordance with their respective licenses.
📖 Citation
If you use these checkpoints in your research, please cite:
bibtex
1@article{koleilat2026medclipseg,
2 title={MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentation},
3 author={Koleilat, Taha and Asgariandehkordi, Hojat and Manzari, Omid Nejati and Barile, Berardino and Xiao, Yiming and Rivaz, Hassan},
4 journal={arXiv preprint arXiv:2602.20423},
5 year={2026}
6}
