IndicTrans2 - INT8 Quantized ONNX Models
Quantized version of AI4Bharat's IndicTrans2 for efficient on-device Indian language translation.
📋 Model Details
Original Model : ai4bharat/indictrans2-indic-en-1B Quantization : INT8 (via ONNX Runtime quantization)Framework : ONNX RuntimeTask : Translation from 22 Indian languages to EnglishUse Case : Offline translation on mobile/edge devices
🗂️ Files Included
Core Models
encoder_int8.onnx
Input: Token IDs [batch, seq_len]
Output: Hidden states [batch, seq_len, 1024]
decoder_int8.onnx
Input: Decoder token IDs + encoder hidden states
Output: Logits [batch, seq_len, vocab_size]
Supporting Files
vocab_src.jsonvocab_tgt.jsonspecial_tokens.json
📊 Compression Stats
Model Original (FP32) Quantized (INT8) Reduction Encoder ~464 MB 116 MB ~75% Decoder ~368 MB 92 MB ~75% Total ~832 MB 208 MB ~75%
🚀 Quick Start
Python (ONNX Runtime)
1 import onnxruntime as ort
2 import json
3 import numpy as np
4
5 # Load models
6 encoder_session = ort . InferenceSession ( "encoder_int8.onnx" )
7 decoder_session = ort . InferenceSession ( "decoder_int8.onnx" )
8
9 # Load vocabularies
10 with open ( "vocab_src.json" ) as f :
11 src_vocab = json . load ( f )
12 with open ( "vocab_tgt.json" ) as f :
13 tgt_vocab = json . load ( f )
14
15 # Tokenize input (add language tags)
16 text = "नमस्ते दुनिया" # Hello world in Hindi
17 tokens = tokenize ( f"<2en> <hin> { text } " , src_vocab )
18 input_ids = np . array ( [ tokens ] , dtype = np . int64 )
19
20 # Run encoder
21 encoder_output = encoder_session . run (
22 [ "hidden_states" ] ,
23 { "input_ids" : input_ids }
24 ) [ 0 ]
25
26 # Autoregressive decoding
27 generated_ids = [ 2 ] # Start token
28 max_length = 50
29
30 for _ in range ( max_length ) :
31 decoder_ids = np . array ( [ generated_ids ] , dtype = np . int64 )
32 logits = decoder_session . run (
33 [ "logits" ] ,
34 {
35 "input_ids" : decoder_ids ,
36 "encoder_hidden_states" : encoder_output
37 }
38 ) [ 0 ]
39
40 next_token = np . argmax ( logits [ 0 , - 1 , : ] )
41 if next_token == 2 : # EOS token
42 break
43 generated_ids . append ( int ( next_token ) )
44
45 # Decode output
46 translation = detokenize ( generated_ids , tgt_vocab )
47 print ( translation ) # "Hello world"
🎯 Supported Languages
Translation from any of these 22 languages to English:
Assamese, Bengali, Bodo, Dogri, Gujarati, Hindi, Kannada, Kashmiri, Konkani, Maithili, Malayalam, Manipuri, Marathi, Nepali, Odia, Punjabi, Sanskrit, Santali, Sindhi, Tamil, Telugu, Urdu
📐 Model Architecture
Input Text (+ lang tags)
↓
Tokenization
↓
Encoder (Transformer, 1B params)
↓ [hidden_states]
Decoder (Autoregressive)
↓
Output Tokens
↓
English Translation⚙️ Performance
Tested on Android (Pixel 7):
Encoder : ~50-150msDecoder (per token): ~20-40msTotal (20 tokens output): ~600ms-1sMemory : ~400MB peak
📝 Language Tags
Use these tags for source language:
<2en> - Translate to English<hin> - Hindi<ben> - Bengali<tam> - Tamil<tel> - Teluguetc.
Example: <2en> <hin> यह एक परीक्षण है
📝 Citation
1 @article{gala2023indictrans,
2 title={IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages},
3 author={Gala, Jay and others},
4 journal={Transactions on Machine Learning Research},
5 year={2023}
6 }
🏗️ Original Creators
📄 License
MIT License (same as original model)
🔗 Related
Quantized for mobile deployment | January 2026
