[!TIP] This model is part of a suite of optimized/quantized ONNX versions of the base model. Other variants in this direction:

FP32 (Full Precision / Base): hari31416/indictrans2-en-indic-dist-200M-ONNX

FP16 (Half Precision): hari31416/indictrans2-en-indic-dist-200M-ONNX-fp16

INT8 (Dynamic Quantization): hari31416/indictrans2-en-indic-dist-200M-ONNX-int8

Q4F16 (4-bit Block Quantization): hari31416/indictrans2-en-indic-dist-200M-ONNX-q4f16 (Current)

ONNX-exported and quantized version of ai4bharat/indictrans2-en-indic-dist-200M for in-browser and local edge inference.

Precision: Q4F16 (4-bit Block Quantization, Lossy)
Description: 4-bit quantization with float16 scale factors and block size of 32. Reduces model size significantly.
Source Pipeline & Details: For pipeline details, benchmarks, and usage instructions, see the indictrans2-onnx-export GitHub repository.

Built for use with Transformers.js and onnxruntime-web in the browser, with fast BPE tokenizer.json files that don't require the SentencePiece WASM runtime.

Performance Visualizations

These charts show overall tradeoffs, language-level parity, and category breakdown.

Performance Tradeoffs & Size Comparison

Compared against the FP32 ONNX oracle on the golden evaluation fixtures.

Format	Model Size	Exact Match (Token)	Exact Match (Text)	SacreBLEU (Raw)	Latency (Mean)	Speedup vs. FP32
FP32	1.70 GB	100.00%	100.00%	100.00	18.3 ms	1.000x
FP16	892.0 MB	99.64%	99.64%	100.00	24.8 ms	0.736x
INT8	452.9 MB	74.36%	74.36%	90.44	13.2 ms	1.594x
Q4F16	623.3 MB	55.18%	55.64%	81.13	27.3 ms	0.705x

Language-Level Parity (Q4F16)

Exact match rates and translation quality (SacreBLEU / chrF) per language pair under this precision:

Language Code	Total Fixtures	Token Match Rate	Text Match Rate	SacreBLEU	SacreBLEU (chrF)
asm_Beng	50	70.0%	70.0%	86.19	94.14
ben_Beng	50	70.0%	72.0%	87.64	95.05
brx_Deva	50	40.0%	40.0%	69.57	87.96
doi_Deva	50	54.0%	54.0%	79.59	89.12
gom_Deva	50	36.0%	36.0%	66.58	86.31
guj_Gujr	50	70.0%	72.0%	88.75	96.23
hin_Deva	50	76.0%	76.0%	91.75	94.89
kan_Knda	50	56.0%	58.0%	80.30	93.59
kas_Arab	50	30.0%	30.0%	61.28	80.62
mai_Deva	50	40.0%	40.0%	72.57	87.77
mal_Mlym	50	60.0%	60.0%	81.49	94.01
mar_Deva	50	56.0%	60.0%	80.08	90.93
mni_Beng	50	30.0%	30.0%	53.98	72.70
npi_Deva	50	66.0%	66.0%	85.66	95.20
ory_Orya	50	64.0%	64.0%	84.09	94.12
pan_Guru	50	70.0%	70.0%	89.89	95.14
san_Deva	50	44.0%	44.0%	63.21	86.22
sat_Olck	50	34.0%	34.0%	54.40	77.94
snd_Arab	50	64.0%	64.0%	85.93	92.71
tam_Taml	50	56.0%	56.0%	82.30	93.43
tel_Telu	50	62.0%	62.0%	80.77	92.73
urd_Arab	50	66.0%	66.0%	88.59	94.56

Category-Level Parity (Q4F16)

Exact match rates and translation quality grouped by category types:

Category	Total Fixtures	Token Match Rate	Text Match Rate	SacreBLEU	SacreBLEU (chrF)
Generic	286	51.8%	51.8%	76.46	88.50
Lexicon	264	51.9%	52.6%	79.25	90.35
Numerals	264	60.6%	61.4%	78.06	91.87
Politics	286	56.6%	57.0%	77.97	89.71

Translation Mismatch Examples

Here is a sample of up to 5 translation mismatches compared to the FP32 oracle. Many mismatches represent minor synonym differences or spacing variations.

Mismatch #1 (Category: Generic)

Source (eng_Latn → asm_Beng): The quick brown fox jumps over the lazy dog.
Expected (FP32): দ্ৰুত বাদামী শিয়ালটোৱে অলস কুকুৰটোৰ ওপৰত ঝাঁপ দিয়ে।
Actual (Q4F16): দ্ৰুত বাদামী শিয়ালটোৱে অলস কুকুৰৰ ওপৰেৰে নামি যায়।

Mismatch #2 (Category: Numerals)

Source (eng_Latn → asm_Beng): Can you please help me find the nearest hospital?
Expected (FP32): আপুনি অনুগ্ৰহ কৰি মোক নিকটতম চিকিৎসালয়খন বিচাৰি উলিওৱাত সহায় কৰিব পাৰিবনে?
Actual (Q4F16): অনুগ্ৰহ কৰি মোক নিকটতম চিকিৎসালয় বিচাৰি উলিওৱাত সহায় কৰিব পাৰিবনে?

Mismatch #3 (Category: Generic)

Source (eng_Latn → asm_Beng): Artificial intelligence is changing the field of education.
Expected (FP32): কৃত্ৰিম বুদ্ধিমত্তাই শিক্ষাৰ ক্ষেত্ৰত পৰিৱৰ্তন কঢ়িয়াই আহিছে।
Actual (Q4F16): কৃত্ৰিম বুদ্ধিমত্তাই শিক্ষাৰ ক্ষেত্ৰ সলনি কৰি আছে।

Mismatch #4 (Category: Generic)

Source (eng_Latn → asm_Beng): The crop yield has improved due to good rainfall.
Expected (FP32): ভাল বৰষুণৰ ফলত শস্যৰ উৎপাদন উন্নত হৈছে।
Actual (Q4F16): ভাল বৰষুণৰ বাবে শস্যৰ উৎপাদন উন্নত হৈছে।

Mismatch #5 (Category: Politics)

Source (eng_Latn → asm_Beng): We need to protect the environment and plant more trees.
Expected (FP32): আমি পৰিৱেশ সুৰক্ষিত কৰিব লাগিব আৰু অধিক গছ ৰোপণ কৰিব লাগিব।
Actual (Q4F16): আমি পৰিৱেশ সুৰক্ষিত কৰিব লাগিব আৰু অধিক গছ-গছনি ৰোপণ কৰিব লাগিব।

Files

encoder_model.onnx (and optional .onnx.data weights sidecar)
decoder_model.onnx and decoder_with_past_model.onnx (share decoder_shared.onnx.data when present)
translate.py — self-contained Python inference helper (see Usage below)
Fast tokenizer config files (tokenizer_src.json, tokenizer_tgt.json, tokenizer_meta.json)
Model configuration configs (config.json, generation_config.json)

Usage Example (Python, onnxruntime)

python

# translate.py is included in this repo alongside the ONNX bundle.
# You can also find it (and read the full source) at:
#   https://github.com/Hari31416/indictrans2-onnx-export/blob/main/src/translate.py

from translate import IndicTransONNX

# Pass a HF repo ID for automatic download, or a local bundle directory path
model = IndicTransONNX("hari31416/indictrans2-en-indic-dist-200M-ONNX-q4f16")
print(model.translate("Who will win the election?", src_lang="eng_Latn", tgt_lang="hin_Deva"))

Required packages:

pip install onnxruntime tokenizers huggingface-hub

License

MIT (preserved from upstream AI4Bharat).