Cohere Labs Tiny Aya is an open weights research release of a pretrained 3.35 billion parameter model optimized for efficient, strong, and balanced multilingual representation across 70+ languages, including many lower-resourced ones. The model is designed to support downstream adaptation, instruction tuning, and local deployment under realistic compute constraints.
For more details about this model family, please check out our blog post and tech report.
Try Cohere Labs Tiny Aya
You can try out Cohere Labs Tiny Aya before downloading the weights in our hosted Hugging Face Space.
Usage
py
1from transformers import AutoTokenizer, AutoModelForCausalLM
23model_id ="CohereLabs/tiny-aya-global"4tokenizer = AutoTokenizer.from_pretrained(model_id)5model = AutoModelForCausalLM.from_pretrained(model_id)67# Format message with the chat template8messages =[{"role":"user","content":"Explica en español qué significa la palabra japonesa 'ikigai' y da un ejemplo práctico."}]9input_ids = tokenizer.apply_chat_template(10 messages,11 tokenize=True,12 add_generation_prompt=True,13 return_tensors="pt",14)1516gen_tokens = model.generate(17 input_ids,18 max_new_tokens=4096,19 do_sample=True,20 temperature=0.1,21 top_p=0.9522)2324gen_text = tokenizer.decode(gen_tokens[0])25print(gen_text)
You can also use the model directly using transformers pipeline abstraction:
Model Architecture: This is an auto-regressive language model that uses an optimized transformer architecture. After pretraining, this model uses supervised fine-tuning (SFT) and preference training to align model behavior to human preferences for helpfulness and safety. The model features three layers with sliding window attention (window size 4096) and RoPE for efficient local context modeling and relative positional encoding. A fourth layer uses global attention without positional embeddings, enabling unrestricted token interactions across the entire sequence.
Languages covered: The model has been trained on 70+ languages, with a focus on: English, Dutch, French, Italian, Portuguese, Romanian, Spanish, Czech, Polish, Ukrainian, Russian, Greek, German, Danish, Swedish, Norwegian, Catalan, Galician, Welsh, Irish, Basque, Croatian, Latvian, Lithuanian, Slovak, Slovenian, Estonian, Finnish, Hungarian, Serbian, Bulgarian, Arabic, Persian, Urdu, Turkish, Maltese, Hebrew, Hindi, Marathi, Bengali, Gujarati, Punjabi, Tamil, Telugu, Nepali, Tagalog, Malay, Indonesian, Vietnamese, Javanese, Khmer, Thai, Lao, Chinese, Burmese, Japanese, Korean, Amharic, Hausa, Igbo, Malagasy, Shona, Swahili, Wolof, Xhosa, Yoruba, and Zulu
Context Length: Tiny Aya supports a context length of 8K & 8K output length.
Language Understanding on CC
Performance Comparison
Generation Quality on Dolly
Usage and Limitations
Intended Usage
Tiny Aya is a family of massively multilingual small language models built to bring capable AI to languages that are often underserved by existing models. The models support languages across Indic, East and Southeast Asian, African, European, and Middle Eastern language families, with a deliberate emphasis on low-resource language performance.
Intended applications include multilingual text generation, conversational AI, summarization, translation and cross-lingual tasks, as well as research in multilingual NLP and low-resource language modeling. The models are also suited for efficient deployment in multilingual regions, helping bridge the digital language divide for underrepresented language communities.
Strengths
Tiny Aya demonstrates strong open-ended generation quality across its full language coverage, with particularly notable performance on low-resource languages. The model performs well on translation, summarization, and cross-lingual tasks, benefiting from training signal shared across language families and scripts.
Limitations
Reasoning tasks. The model's strongest performance is on open-ended generation and conversational tasks. Chain-of-thought reasoning tasks such as multilingual math (MGSM) are comparatively weaker.
Factual knowledge. As with any language model, outputs may contain incorrect or outdated statements, particularly in lower-resource languages with thinner training data coverage.
Uneven resource distribution. High-resource languages benefit from richer training signal and tend to exhibit more consistent quality across tasks. The lowest-resource languages in the model's coverage may show greater variability, and culturally specific nuance, sarcasm, or figurative language may be less reliably handled in these languages.
Task complexity. The model performs best with clear prompts and instructions. Highly complex or open-ended reasoning, particularly in lower-resource languages, remains challenging.
Model Card Contact
For errors or additional questions about details in this model card, contact [labs@cohere.com].
Terms of Use:
We hope that the release of this model will make community-based research efforts more accessible, by releasing the weights of a highly performant 111 billion parameter model to researchers all over the world. This model is governed by a CC-BY-NC License (Non-Commercial) with an acceptable use addendum, and also requires adhering to Cohere Lab's Acceptable Use Policy. If you are interested in commercial use, please contact Cohere’s Sales team.
@misc{salamanca2026tinyayabridgingscale,
title={Tiny Aya: Bridging Scale and Multilingual Depth},
author={Alejandro R. Salamanca and Diana Abagyan and Daniel D'souza and Ammar Khairi and David Mora and Saurabh Dash and Viraat Aryabumi and Sara Rajaee and Mehrnaz Mofakhami and Ananya Sahu and Thomas Euyang and Brittawnya Prince and Madeline Smith and Hangyu Lin and Acyr Locatelli and Sara Hooker and Tom Kocmi and Aidan Gomez and Ivan Zhang and Phil Blunsom and Nick Frosst and Joelle Pineau and Beyza Ermis and Ahmet Üstün and Julia Kreutzer and Marzieh Fadaee},
year={2026},
eprint={2603.11510},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2603.11510},
}
