This is an instruction-tuned (SFT) language model for Chichewa (nya). It was created by applying supervised fine-tuning on top of the BYOL Chichewa 12b CPT checkpoint, using translated instruction-following data (SmolTalk2 + AYA) generated via the BYOL framework.

This is an intermediate checkpoint used to produce the merged model. For best results, use the merged variant instead, which combines the language knowledge from CPT with the instruction-following ability from this model.

Usage

Base model: <a href="https://huggingface.co/google/gemma-3-12b-pt" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">google/gemma-3-12b-pt
Language: Chichewa (nya)
Training stage: Instruction Tuning (SFT)
License: <a href="https://ai.google.dev/gemma/terms" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">Gemma Terms of Use (derived from Gemma 3)
Paper: <a href="https://arxiv.org/abs/2601.10804" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">BYOL: Bring Your Own Language Into LLMs
Code: <a href="https://github.com/microsoft/byol" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">github.com/microsoft/byol

pip install -U transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "ai-for-good-lab/byol-nya-12b-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", dtype=torch.bfloat16)

# Chat inference
messages = [{"role": "user", "content": "Tandiuzeni za dziko la Malawi."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True, return_dict=True).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation

bibtex

@article{zamir2026byolbringlanguagellms,
    title={BYOL: Bring Your Own Language Into LLMs},
    author={Syed Waqas Zamir and Wassim Hamidouche and Boulbaba Ben Amor and Luana Marotti and Inbal Becker-Reshef and Juan Lavista Ferres},
    year={2026},
    journal={arXiv:2601.10804},
    url={https://arxiv.org/abs/2601.10804},
}