A LoRA adapter on top of facebook/nllb-200-distilled-600M for Swahili → Kalenjin translation. Kalenjin is a Nilotic language with ~5M speakers across the Kenyan Rift Valley, parts of Tanzania, Uganda, and the diaspora; NLLB-200 ships no Kalenjin code, so this adapter decodes into the unused luo_Latn token slot.

Designed to run as the second leg of an English → Swahili → Kalenjin cascade. NLLB-200 base handles the EN→SW leg out of the box; this adapter handles the SW→KL leg.

Quickstart

python

import torch
from transformers import AutoModelForSeq2SeqLM, NllbTokenizerFast
from peft import PeftModel

base = AutoModelForSeq2SeqLM.from_pretrained(
    "facebook/nllb-200-distilled-600M", torch_dtype=torch.float32
)
model = PeftModel.from_pretrained(base, "Tonykip/swahili-kalenjin-mt-lora")
model.eval()

tok = NllbTokenizerFast.from_pretrained("facebook/nllb-200-distilled-600M")
tok.src_lang = "swh_Latn"
inputs = tok("Habari ya asubuhi, rafiki.", return_tensors="pt", truncation=True)

# Decode into Kalenjin via the luo_Latn token slot
luo_id = tok.convert_tokens_to_ids("luo_Latn")
out = model.generate(
    **inputs,
    forced_bos_token_id=luo_id,
    max_length=256,
    num_beams=5,
    early_stopping=True,
)
print(tok.batch_decode(out, skip_special_tokens=True)[0])

Two-leg cascade (EN → SW → KL)

python

# Leg 1: EN -> SW (base NLLB, adapter disabled)
with model.disable_adapter():
    tok.src_lang = "eng_Latn"
    inputs = tok("Tell my brother I'll visit next week.", return_tensors="pt")
    swh = model.generate(
        **inputs,
        forced_bos_token_id=tok.convert_tokens_to_ids("swh_Latn"),
        max_length=256, num_beams=5, early_stopping=True,
    )
    swahili = tok.batch_decode(swh, skip_special_tokens=True)[0]

# Leg 2: SW -> KL (adapter active, decoding to luo_Latn slot)
tok.src_lang = "swh_Latn"
inputs = tok(swahili, return_tensors="pt")
kln = model.generate(
    **inputs,
    forced_bos_token_id=tok.convert_tokens_to_ids("luo_Latn"),
    max_length=256, num_beams=5, early_stopping=True,
)
kalenjin = tok.batch_decode(kln, skip_special_tokens=True)[0]

Training

Base: facebook/nllb-200-distilled-600M
Adapter: LoRA, rank 64, alpha 128, dropout 0.05
Targets: q_proj, k_proj, v_proj, out_proj, fc1, fc2
Schedule: 42 epochs · cosine LR with warmup_ratio=0.05 · optim=adamw_8bit
Token hijack: Kalenjin produced via the luo_Latn slot, the closest Nilotic neighbor NLLB ships with

Evaluation

On a 250-row locked held-out set drawn from thinkKenya/kenyan-low-resource-language-data (kln_swa config, test split):

Metric	Score
chrF++	58.79
BLEU	~6

Generation params used for the score: num_beams=5, max_length=256, early_stopping=True, forced_bos_token_id=luo_Latn.

Intended use

Built for diaspora speakers, heritage learners, and anyone reconnecting with Kalenjin through written or spoken language work. Ships open-weights so research builds can reproduce, audit, and extend.

This is a first Kalenjin MT release. Expect register and dialect mismatches on conversational vs literary inputs. Long literary prose (Anv-ke style) generalizes better than informal speech.

Citation

bibtex

@misc{kipkemboi2026kalenjincascade,
  title  = {A Two-Leg Cascade for English → Kalenjin Machine Translation},
  author = {Tony Kipkemboi},
  year   = {2026},
  note   = {chamgei.labs working notes},
  url    = {https://chamgei.com}
}

Issue tracker / source: tonykipkemboi/whisper-kalenjin-lora