This model checkpoint contains the seg probes for CLIP-ConvNeXT-XXL Llama-3-8b based LLaVA-1.5 model after the PT, VPT, and IFT stages, i.e., trained on the LLaVA-558K, ALLaVA-Caption-663K, and LLaVA-665K dataset. Please refer to documentation for more details.
If you found our work useful in your research, please consider starring ⭐ us on GitHub and citing 📚 us in your research!
@article{jain2024ola_vlm,
title={{OLA-VLM: Elevating Visual Perception in Multimodal LLMs with Auxiliary Embedding Distillation}},
author={Jitesh Jain and Zhengyuan Yang and Humphrey Shi and Jianfeng Gao and Jianwei Yang},
journal={arXiv},
year={2024}
}
