TowerVision is a family of open-source multilingual vision-language models with strong capabilities optimized for a variety of vision-language use cases, including image captioning, visual understanding, summarization, question answering, and more. TowerVision excels particularly in multimodal multilingual translation benchmarks and culturally-aware tasks, demonstrating exceptional performance across 20 languages and dialects.
This model card covers the TowerVision family, including the 2B and 9B parameter versions, both in their instruct-tuned (it) and pretrained (pt) variants, with the latter not undergoing instruction tuning.
Model Family: TowerVision (2B, 9B variants)
Context length: 8192 tokens
Languages: 20+ languages including European, Asian, and other language families
1mport torch
2from transformers import AutoProcessor, LlavaOnevisionForConditionalGeneration
34# Load the model in half-precision5model = LlavaOnevisionForConditionalGeneration.from_pretrained(6"utter-project/TowerVideo-2B",7 device_map="auto"8)910processor = AutoProcessor.from_pretrained(11"utter-project/TowerVideo-2B"12)1314# Use your local video15video_path ="your_video_path.mp4"1617# Conversation using the same template18conversation =[19{20"role":"user",21"content":[22{"type":"video","path": video_path},23{"type":"text","text":"\n<video>\nIWhat is the video about?"},24],25},26]2728# Apply the chat template29inputs = processor.apply_chat_template(30 conversation,31 num_frames=8,32 add_generation_prompt=True,33 tokenize=True,34 return_dict=True,35 add_special_tokens=True,# ensures <video> token is inserted36 return_tensors="pt"37).to(model.device, torch.float16)3839# Generate response40out = model.generate(**inputs, max_new_tokens=60)4142# Decode output43decoded = processor.batch_decode(44 out,45 skip_special_tokens=True,46 clean_up_tokenization_spaces=True47)4849print(decoded)
Model Details
Input: Model accepts input text, images and video.
Output: Model generates text in multiple languages.
Model Architecture: TowerVideo uses a multilingual image-language model based on Tower-Plus (2B and 9B parameters), paired with SigLIP2-patch14-384 vision encoder through a multimodal adapter for vision-language understanding.
Recommended Precision: We recommend using bfloat16 precision for optimal performance and memory efficiency when running TowerVision models.
Languages Covered: The model has been trained on 20 languages and dialects:
Asian languages: Chinese (Simplified & Traditional), Japanese, Korean, Hindi
Other languages: Russian, Ukrainian
Key Strengths:
🏆 Exceptional performance on culturally-aware benchmarks with deep understanding of cultural contexts and visual nuances
📊 Strong cross-lingual transfer capabilities across diverse vision-language tasks
Training Data
TowerVision models are trained on a video/text subset of VisionBlocks, a comprehensive multilingual vision-language dataset comprising 6.31M samples across diverse categories:
Collection Types: Human-annotated, synthetically generated, and professionally translated data ensuring high quality and cultural diversity across 20+ languages.
📊 Benchmark Coverage: Our models are evaluated across diverse multilingual vision-language tasks, demonstrating strong cross-lingual transfer capabilities and exceptional performance in culturally-aware benchmarks.
Citation
If you find TowerVideo useful in your research, please consider citing the following paper:
bibtex
1@misc{viveiros2025towervisionunderstandingimprovingmultilinguality,
2 title={TowerVision: Understanding and Improving Multilinguality in Vision-Language Models},
3 author={André G. Viveiros and Patrick Fernandes and Saul Santos and Sonal Sannigrahi and Emmanouil Zaranis and Nuno M. Guerreiro and Amin Farajian and Pierre Colombo and Graham Neubig and André F. T. Martins},
4 year={2025},
5 eprint={2510.21849},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/2510.21849},
9}
Model Card Contact
For errors or additional questions about details in this model card, contact the research team.
Acknowledgments
TowerVision builds upon the excellent work of:
LLaVA-NeXT for the foundational vision-language architecture
TowerVision-9B vision-language model with multilingual capabilities
