The original model was trained by DSFSI (Data Science for Social Impact research group, University of Pretoria), a fine-tune of facebook/m2m100_418M. Licence: Apache-2.0, as declared by the original repository. This repository only converts the weights to ONNX. All credit for the model belongs to DSFSI.

Export

optimum-cli export onnx --model dsfsi/m2m100_418m-nso-eng \
  --task text2text-generation-with-past --no-post-process <outdir>

The int8 quantized files needed the .onnx.data -> .onnx_data rename and proto location rewrite (encoder, decoder and decoder-with-past, plus one Constant node tensor attribute per file). Verified with a real load from a cache cleared beforehand.

Files

Path	Precision	Size
`*.onnx` (root)	fp32	4.6 GB
`int8/.onnx` + `int8/.onnx_data`	int8 dynamic	3.0 GB

Parity

10 general-domain Northern Sotho sentences (source language for this nso->en pair), PyTorch original vs ONNX, exact string match against AutoModelForSeq2SeqLM.generate().

yaml

parity:
  sample_size: 10
  metric: exact_match
  fp32_greedy: 1.00   # 10/10
  fp32_beam4:  1.00   # 10/10
  int8_greedy: 0.80   # 8/10
  int8_beam4:  0.70   # 7/10

Decoding	fp32 (n=10)	int8 (n=10)
greedy	100% (10/10)	80% (8/10)
beam=4	100% (10/10)	70% (7/10)

fp32 is an exact reproduction of the original model. Note that the base model's own English output is already somewhat rough on this reverse direction regardless of precision (e.g. "The weather is the easiest at the time." for a sentence meaning "the weather is very nice today") - that is inherited from the DSFSI checkpoint, not introduced by this conversion. int8 disagreements are mostly paraphrases, but one is a genuine pronoun error worth flagging honestly: "Three years ago she worked on the project." (ref) vs "Three years ago he worked on the project." (int8).

Caveat on sample size: n=10 exact-match is a small, noisy estimate, not a FLORES-scale benchmark. A separate measurement campaign on this project found n=5 exact-match numbers for another model swing from 100% to 14% when re-measured at n=100 on FLORES-200. Treat the percentages above as an initial signal only - a proper FLORES + chrF evaluation supersedes it wherever available.

Selecting the language

Northern Sotho's M2M100 language code is ns (not nso). The target is not baked into forced_bos_token_id in generation_config.json for this checkpoint - you must set it explicitly:

python

tokenizer.src_lang = "ns"
forced_bos_token_id = tokenizer.lang_code_to_id["en"]
model.generate(**inputs, forced_bos_token_id=forced_bos_token_id, ...)

Usage

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

repo = "TigreGotico/m2m100_418m-nso-eng-onnx"
tok = AutoTokenizer.from_pretrained(repo)
tok.src_lang = "ns"
model = ORTModelForSeq2SeqLM.from_pretrained(repo, use_cache=True, use_merged=False)

enc = tok("Boso bo bonolo kudu mo lebakeng le.", return_tensors="pt")
tgt_id = tok.lang_code_to_id["en"]
out = model.generate(**enc, forced_bos_token_id=tgt_id, num_beams=4, max_new_tokens=64)
print(tok.batch_decode(out, skip_special_tokens=True)[0])

For the int8 build, pass subfolder="int8".