This repo contains the weights of VW-LMM-Mistral-7b proposed in paper "Multi-modal Auto-regressive Modeling via Visual Words"
For specific usage and chat templates, please refer to our project repo
https://github.com/pengts/VW-LMMModel type:
VW-LMM is an open-source chatbot trained by fine-tuning LLM on multimodal instruction-following data.
It is an auto-regressive language model, based on the transformer architecture.
Base LLM:
mistralai/Mistral-7B-Instruct-v0.2
If you find our paper and code useful in your research, please consider giving a star :star: and citation :pencil:.
1@misc{peng2024multimodal,
2 title={Multi-modal Auto-regressive Modeling via Visual Words},
3 author={Tianshuo Peng and Zuchao Li and Lefei Zhang and Hai Zhao and Ping Wang and Bo Du},
4 year={2024},
5 eprint={2403.07720},
6 archivePrefix={arXiv},
7 primaryClass={cs.CV}
8}
