This project builds a bidirectional English–Tigrinya translation model by taking a pretrained multilingual NLLB-200 checkpoint and transplanting its tokenizer for the MoVoC_Tok (a 120,000-vocabulary SentencePiece Unigram tokenizer shared across Amharic/Tigrinya/Tigre/Ge'ez/English, built earlier in this line of work) — then fully fine-tuning on a cleaned, deduplicated, multi-source parallel corpus. It extends the methodology of our own prior paper, Low-Resource English-Tigrinya MT: Leveraging Multilingual Models, Custom Tokenizers, and Clean Evaluation Benchmarks (Teklehaymanot, Gidey, Nejdl — LREC 2026), applying it to this newer, larger, 5-language-shared tokenizer instead of a narrower one.

Code: github.com/hailaykidu/EnTiMT

Pipeline

01_collection/       -> raw parallel data from every verified public source
02_cleaning/          -> normalize, filter, dedup (exact + MinHash near-dup), merge
03_tokenizer_integration/ -> transplant MoVoC_Tok into NLLB-200-distilled-600M
04_training/          -> bidirectional Seq2SeqTrainer fine-tune (SLURM)
05_evaluation/        -> BLEU / chrF / COMET on a held-out gold benchmark

1. Data collection

Every source below was verified by actually downloading it (OPUS API, GitHub clone, or reuse of an already-local repo from this session) -- see 01_collection/SOURCES.md in the code repo for the full table with license and exact pair counts. In short:

Source	Pairs	Quality
OPUS NLLB (mined)	1,398,173	noisy, web-mined
Travis Foundation Tigrinya-Parallel-Corpus	126,930	human-collected
OPUS CCAligned	7,658	noisy, web-mined
OPUS tico-19	6,142	professionally translated (COVID-19 domain)
HornMT	2,030	human-translated news, multi-way parallel
OPUS Tatoeba	74	human, tiny

FLORES-200 devtest (tir_Ethi/eng_Latn) is fetched separately and used only as the final held-out evaluation benchmark -- it is never mixed into the training pool.

2. Cleaning

clean_corpus.py normalizes (NFC, control-char/whitespace cleanup), strips mined-bitext artifacts found in the raw NLLB data (Bible verse numbers, [alt1//alt2] inline alternate-phrasing brackets), filters by length and length-ratio, keeps only Tigrinya-script-consistent pairs, then deduplicates (exact + MinHash near-duplicate over word 3-shingles, reusing the same approach as the MoVoC_Tok project's corpus cleaning).

One real bug caught and fixed during this step: Python's str.splitlines() splits on more than \n (including U+2028 LINE SEPARATOR, present 84 times in the raw NLLB English file and only 4 times in its Tigrinya counterpart) -- using it to read the two sides of a parallel file independently silently desynced their line-for-line alignment partway through the corpus. Switched to plain \n-splitting, which matches how the files are actually newline-delimited, and re-verified alignment by spot-checking matched line indices before proceeding.

3. Tokenizer transplant

transplant_tokenizer.py replaces facebook/nllb-200-distilled-600M's own ~256k-token tokenizer with MoVoC_Tok, and re-initializes the embedding matrix with a FOCUS-style informed initialization rather than random init:

Tokenize the cleaned training corpus with MoVoC_Tok itself, so the auxiliary embedding model sees the exact subword pieces this vocabulary produces.
Train a FastText skip-gram model directly on that tokenized text, at dim == 1024 (NLLB-200-distilled-600M's d_model) -- no projection needed since dimensions already match.
Use those vectors as the new embedding matrix, written in place via .data indexing after resize_token_embeddings(), which preserves tied encoder/decoder-input/LM-head weights.
The pretrained transformer body (attention/FFN weights) is kept as-is -- that is the actual transfer-learning payload; only the embedding layer changes.

Bidirectionality is handled with direction tags (>>tir<< / >>eng<<) prepended to the encoder input.

4. Training

train_mt.py builds a bidirectional training set (every pair trains both en->ti and ti->en, so 1,140,309 cleaned pairs become 2,280,618 training examples) and fully fine-tunes with Seq2SeqTrainer (no frozen layers -- see the tokenizer-transplant section above for why).

Training configuration:

Setting	Value
Base checkpoint	`facebook/nllb-200-distilled-600M` (tokenizer/embeddings replaced, body kept)
Train / dev examples	2,280,618 / 4,000 (bidirectional; dev held out from cleaning, distinct from FLORES-200)
Epochs	3
Per-device train / eval batch size	16 / 16
Gradient accumulation	4 (effective batch size 64)
Learning rate	3e-5
Max sequence length	128 tokens
Precision	bf16
Eval / save interval	every 2,000 steps
Model selection	`load_best_model_at_end=True`, `metric_for_best_model="eval_loss"`
Seed	42
Direction tags	`>>tir<<` (target=Tigrinya), `>>eng<<` (target=English), prepended to encoder input
Steps / wall time	106,905 steps / 13h21m
Final train loss (mean of all logged steps)	4.696

5. Evaluation

evaluate.py reports BLEU and chrF (via sacrebleu) and COMET (via unbabel-comet, Unbabel/wmt22-comet-da) on the held-out FLORES-200 devtest set (1,013 sentences, zero overlap with training data), in both directions, alongside qualitative before/after examples.

Results (real, FLORES-200 devtest, 1,013 sentences)

Direction	BLEU	chrF	COMET
en → ti	0.133	4.99	0.416
ti → en	0.899	15.91	0.374

These scores are poor, and this is reported honestly rather than omitted. Qualitative inspection of the FLORES-200 output shows severe repetition/degeneration under greedy decoding, especially on longer source sentences — e.g. a real en→ti output for a ~20-word input:

"ኣብ'ዚ እዋን'ዚ ኣብ'ዚ እዋን'ዚ ኣብ'ዚ እዋን'ዚ ኣብ'ዚ እዋን'ዚ ኣብ'ዚ እዋን'ዚ'ዚ'ዚ'ዚ ..."

and another where "ዶክተር" ("doctor") repeats over 100 times in place of an actual translation. This pattern — the model locking into a short repeated n-gram instead of producing a full translation — appears throughout the devtest set, not just on cherry-picked examples (see 05_evaluation/eval_report.json in the GitHub repo for unedited qualitative samples). Likely contributing factors, not yet isolated: greedy decoding with no repetition penalty, and/or insufficient training given the size of the newly-transplanted embedding matrix relative to available compute.

Do not use this checkpoint for anything beyond experimentation as-is. Candidate fixes not yet tried: beam search / repetition penalty at generation time, additional training epochs, or a smaller learning rate specifically for the embedding layer.

Usage

python

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model = AutoModelForSeq2SeqLM.from_pretrained("Hailay/entimt-en-tigrinya-mt")
tokenizer = AutoTokenizer.from_pretrained("Hailay/entimt-en-tigrinya-mt")

text = ">>tir<< The weather is nice today."
inputs = tokenizer(text, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Given the repetition issue above, num_beams>1 and a repetition_penalty (e.g. no_repeat_ngram_size=3, repetition_penalty=1.3) are recommended over plain greedy decoding, though this hasn't been systematically re-evaluated.

Limitations

The bulk of training volume comes from web-mined NLLB bitext, which is noisier than human-translated data even after cleaning -- quality is upper-bounded by that source's alignment accuracy.
The FastText-based embedding initialization is a simplified, self-contained approximation of the FOCUS technique (trained on this project's own corpus rather than a large general-purpose auxiliary embedding space); it gives every subword piece a distributionally grounded starting point, not a semantically "solved" one.
Tigrinya has real dialectal variation (Eritrean vs. Ethiopian) not explicitly modeled or balanced for in the training data.

Installation

pip install transformers torch sentencepiece

Citation

This artifact uses the MoVoC-Tok tokenizer introduced in Teklehaymanot et al. (2025). Please cite:

bibtex

@inproceedings{teklehaymanot2025movoc,
  title     = {MoVoC: Morphology-Aware Subword Construction for Ge'ez Script Languages},
  author    = {Teklehaymanot, Hailay Kidu and Fazlija, Dren and Nejdl, Wolfgang},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2025},
  year      = {2025},
  url       = {https://arxiv.org/abs/2509.08812}
}

Hailay/entimt-en-tigrinya-mt

EnTiMT: English ↔ Tigrinya Machine Translation with a Transplanted MoVoC_Tok