The original model was trained by DSFSI (Data Science for Social Impact research group, University of Pretoria), a fine-tune of facebook/nllb-200-distilled-600M on the afriscience-mt dataset. Licence: Apache-2.0, as declared by the original repository. This repository only converts the weights to ONNX. All credit for the model belongs to DSFSI.

Export

optimum-cli export onnx --model dsfsi/nllb_200_distilled_600m-yor-eng \
  --task text2text-generation-with-past --no-post-process <outdir>

The int8 quantized files needed the .onnx.data -> .onnx_data rename and proto location rewrite (encoder, decoder and decoder-with-past, plus one Constant node tensor attribute per file). Verified with a real load from a cache cleared beforehand.

Files

Path	Precision	Size
`.onnx` + `.onnx_data` (root)	fp32	~7 GB
`int8/.onnx` + `int8/.onnx_data`	int8 dynamic	~5.5 GB

Parity

10 general-domain Yoruba sentences (source language for this yo->en pair), PyTorch original vs ONNX, exact string match against AutoModelForSeq2SeqLM.generate().

yaml

parity:
  sample_size: 10
  metric: exact_match
  fp32_greedy: 1.00   # 10/10
  fp32_beam4:  1.00   # 10/10
  int8_greedy: 0.80   # 8/10
  int8_beam4:  0.90   # 9/10

Decoding	fp32 (n=10)	int8 (n=10)
greedy	100% (10/10)	80% (8/10)
beam=4	100% (10/10)	90% (9/10)

fp32 is an exact reproduction of the original model. int8 disagreements were inspected by hand: most are paraphrases, but one is a genuine semantic error worth flagging honestly rather than downplaying - for "Ìjọba kéde àwọn ìlànà ọrọ̀-ajé tuntun lánàá." (source says "yesterday") the reference correctly outputs "...yesterday." while int8 at both greedy and beam=4 outputs "...today." This is a real, repeatable temporal error introduced by int8 quantization on this input, not noise - treat int8 output involving time references on this model with some caution.

Caveat on sample size: n=10 exact-match is a small, noisy estimate, not a FLORES-scale benchmark. A separate measurement campaign on this project found n=5 exact-match numbers for another model swing from 100% to 14% when re-measured at n=100 on FLORES-200. Treat the percentages above as an initial signal only - a proper FLORES + chrF evaluation supersedes it wherever available.

Selecting the language

This is an NLLB checkpoint, not M2M100: the target language is not baked into forced_bos_token_id in generation_config.json - you must pass it explicitly at generate time using NLLB's flores_code language tags (yor_Latn, eng_Latn, not plain ISO 639-1 codes):

python

tokenizer.src_lang = "yor_Latn"
forced_bos_token_id = tokenizer.convert_tokens_to_ids("eng_Latn")
model.generate(**inputs, forced_bos_token_id=forced_bos_token_id, ...)

Usage

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

repo = "TigreGotico/nllb_200_distilled_600m-yor-eng-onnx"
tok = AutoTokenizer.from_pretrained(repo)
tok.src_lang = "yor_Latn"
model = ORTModelForSeq2SeqLM.from_pretrained(repo, use_cache=True, use_merged=False)

enc = tok("Ojú-ọjọ́ dára gan-an lọ́jọ́ òní.", return_tensors="pt")
tgt_id = tok.convert_tokens_to_ids("eng_Latn")
out = model.generate(**enc, forced_bos_token_id=tgt_id, num_beams=4, max_new_tokens=64)
print(tok.batch_decode(out, skip_special_tokens=True)[0])
# -> The weather was very good today.

For the int8 build, pass subfolder="int8".