Quantized ONNX speech recognition models for fully offline, on-device use with
sherpa-onnx — including on budget Android
phones with no internet connection required at inference time. Ten Indian languages
from AI4Bharat's IndicConformer, English from
sherpa-onnx's own pretrained NeMo fast-conformer CTC model, and two Hinglish
(Hindi-English code-switched) models built on fine-tuned Whisper checkpoints for
speech that naturally mixes both languages mid-sentence.
Purpose
The original AI4Bharat checkpoints are excellent, but built for server-side/PyTorch
inference and aren't practical to ship inside a mobile app. This repo re-packages them
(and a matching English model) as small, quantized ONNX files that:
Run entirely on-device, with no server, no API key, and no internet needed once
downloaded.
Are small enough (~150-200MB each) to download and run on budget Android hardware.
Work directly with sherpa-onnx's Kotlin/Java API for Android apps, or its Python
API for anything else.
This is a community effort — the goal is to make it easy for any Indian-language app
(subtitle generators, voice assistants, accessibility tools, etc.) to add real offline
speech recognition without needing to repeat this export/quantization work themselves.
Languages available
All 11 languages below are exported, quantized, and validated end-to-end (reference
transcript -> fp32 ONNX -> int8 ONNX -> sherpa-onnx, on both desktop and real Android
hardware):
Code
Language
en
English
hi
Hindi
as
Assamese
gu
Gujarati
pa
Punjabi
bn
Bengali
ta
Tamil
te
Telugu
mr
Marathi
kn
Kannada
ml
Malayalam
Hinglish (Hindi-English code-switched)
Two additional models for natural code-switched speech — a speaker mixing Hindi
and English mid-sentence, the way people actually talk in India. These are a
different model family from the 11 languages above (Whisper-based, not Conformer),
built on Oriserve's Whisper-Hindi2Hinglish
fine-tunes, and output Romanized Hinglish text directly (e.g. "Doston, nested aur multilevel if statement ke spoken tutorial mein aapka svaagat hai.") rather than
Devanagari.
Variant
Base
Params
Common Voice WER
FLEURS WER
Indic-Voices WER
int8 size
hi-hinglish-apex
Whisper Large-V3 (pruned decoder)
807M
36.0%
29.8%
47.6%
~1.0 GB
hi-hinglish-swift
Whisper-base
72.6M
38.7%
35.1%
65.2%
~120-150 MB
Apex is meaningfully more accurate, especially on noisy/conversational audio
(Indic-Voices), at the cost of a much larger download. Swift is far smaller and
faster, trading some accuracy for that. Both were validated by ear against real
Hinglish audio clips, not just benchmark numbers — see "How these were made" below.
Repo layout
tokens.txt <- shared across all 10 AI4Bharat Indic languages
hi/model.int8.onnx
as/model.int8.onnx
gu/model.int8.onnx
pa/model.int8.onnx
bn/model.int8.onnx
ta/model.int8.onnx
te/model.int8.onnx
mr/model.int8.onnx
kn/model.int8.onnx
ml/model.int8.onnx
en/model.int8.onnx
en/tokens.txt <- English has its own, unrelated vocabulary
hi-hinglish-apex/encoder.int8.onnx
hi-hinglish-apex/decoder.int8.onnx
hi-hinglish-apex/tokens.txt
hi-hinglish-swift/encoder.int8.onnx
hi-hinglish-swift/decoder.int8.onnx
hi-hinglish-swift/tokens.txt
The root tokens.txt only needs to be downloaded once for the Indic languages — it is
identical across all ten (verified directly, not assumed). English is a completely
different model family with its own vocabulary, so it ships with its own tokens.txt
inside en/. The Hinglish models use a different repo layout entirely — Whisper's
encoder/decoder are separate ONNX graphs (not one combined model.int8.onnx), each
with its own tokens.txt, so all three files must be downloaded together per variant.
Both encoder and decoder are quantized int8 files that are self-contained (dynamic
quantization doesn't need external weight data even though the intermediate fp32
export did), so no .weights sidecar file is needed or produced.
How to use it
Python (sherpa-onnx)
import sherpa_onnx
# Indic language example (shared tokens.txt)
recognizer = sherpa_onnx.OfflineRecognizer.from_nemo_ctc(
model="hi/model.int8.onnx", # swap for any Indic language folder above
tokens="tokens.txt",
num_threads=2,
decoding_method="greedy_search",
)
# English example (its own tokens.txt)
recognizer_en = sherpa_onnx.OfflineRecognizer.from_nemo_ctc(
model="en/model.int8.onnx",
tokens="en/tokens.txt",
num_threads=2,
decoding_method="greedy_search",
)
stream = recognizer.create_stream()
stream.accept_waveform(16000, audio) # audio: float32 numpy array, 16kHz mono
recognizer.decode_stream(stream)
print(stream.result.text)
# Hinglish example (Whisper-based, not NeMo CTC -- different loader)
recognizer_hinglish = sherpa_onnx.OfflineRecognizer.from_whisper(
encoder="hi-hinglish-apex/encoder.int8.onnx", # or hi-hinglish-swift/...
decoder="hi-hinglish-apex/decoder.int8.onnx",
tokens="hi-hinglish-apex/tokens.txt",
num_threads=2,
decoding_method="greedy_search",
language="hi",
task="transcribe",
)
stream = recognizer_hinglish.create_stream()
stream.accept_waveform(16000, audio)
recognizer_hinglish.decode_stream(stream)
print(stream.result.text) # Romanized Hinglish output
Android (Kotlin)
Use sherpa-onnx's Android AAR and its OfflineRecognizer / OfflineNemoEncDecCtcModelConfig
API — the Kotlin API mirrors the Python one closely. Download the model + matching
tokens file for a language to app-writable storage at runtime, and point the config at
their file paths. No bundled assets or special permissions needed — this is exactly
how the reference Android app (SRTConversionAPP) uses this repo: every language,
including English, is downloaded on demand rather than shipped in the app itself, to
keep install size small.
Downloading files directly
Every file is a plain public HTTPS download, no auth required:
Source: AI4Bharat's indicconformer_stt_<lang>_hybrid_ctc_rnnt_large checkpoints
(Conformer-Large, hybrid CTC+RNNT, 120M params, shared multilingual aggregate
tokenizer across 22 Indian languages).
Exported to ONNX using the CTC decoding head only (not RNNT — much cheaper to run
on mobile), with language-specific masking baked directly into the exported graph,
so any downstream consumer can do plain greedy argmax decoding safely without
needing to know about the shared multilingual vocabulary.
Quantized with onnxruntime's dynamic quantization, restricted to MatMul ops only
(int8). Full-model int8 and fp16 were both tested and rejected for hurting
accuracy; MatMul-only int8 gives roughly 60% size reduction with no measurable
quality loss.
Validated at every stage: PyTorch reference transcript -> fp32 ONNX -> int8 ONNX ->
sherpa-onnx (Python) -> sherpa-onnx (Kotlin, on real Android hardware).
English:
Source: sherpa-onnx's own pretrained sherpa-onnx-nemo-fast-conformer-ctc-en-24500
model (NeMo fast-conformer, CTC) — already a valid ONNX export, so no PyTorch
export or masking step was needed, just quantization.
Quantized with the same MatMul-only int8 recipe as the Indic models (~458MB fp32 ->
~175MB int8), then validated against sherpa-onnx's own bundled test audio.
Hinglish (Apex / Swift):
Source: Oriserve's Whisper-Hindi2Hinglish-Apex
(Whisper Large-V3 base, pruned to a 4-layer decoder, 807M params, fine-tuned on
~700 hours of Indian-accented Hindi/Hinglish audio) and
Whisper-Hindi2Hinglish-Swift (Whisper-base, 72.6M params, ~550 hours). Both
released by Oriserve under Apache 2.0.
Published in Hugging Face transformers weight format (safetensors), not the raw
OpenAI Whisper checkpoint format that ONNX export tooling expects — converted using
each repo's own included convert_hf2openai.json key-rename map, then re-assembled
into an OpenAI-format checkpoint (dims + model_state_dict) and verified to load
cleanly via whisper.load_model() before export.
Exported to ONNX using sherpa-onnx's scripts/whisper/export-onnx-with-attention.py
(from k2-fsa/sherpa-onnx PR #2945),
patched the same way as the base export script (custom checkpoint support, n_mels
read from the model's own config, legacy dynamo=False tracer), plus a per-model
entry in its ALIGNMENT_HEADS table discovered by running find_alignment_heads.py
against each checkpoint — these fine-tunes aren't official OpenAI releases, so their
best cross-attention heads for DTW alignment aren't in the pre-baked table and had to
be found empirically. This produces a decoder with an extra cross_attention_weights
output, enabling genuine per-token timestamps (not just transcription text).
Quantized with the same MatMul-only int8 recipe as the other models.
Validated by ear against real Hinglish audio (not just WER benchmarks) using
sherpa-onnx's OfflineRecognizer.from_whisper() Python API — output was compared
word-for-word against the actual spoken reference text, and per-token timestamps
were checked directly for correct pacing (no repeated or out-of-order values).
Testing / sample audio
If you want real Indian-language audio clips to test these models with, the
Indian Languages Audio Dataset on Kaggle
is a useful public source of labeled multilingual speech samples across several of
the languages covered here.
Reproduce this yourself / convert more languages
Every notebook used to produce a model in this repo is included here, at the repo
root — not just described, actually runnable.
AI4Bharat Indic languages:ai4bharat_export_pipeline.ipynb.
To convert a language not yet listed above, open it in Google Colab, set the runtime
to CPU, change the LANG_CODE variable near the top to the language you want, and run
the cells top to bottom. Every step — model export, masking, quantization, and
validation against sherpa-onnx — is already worked out and documented inline.
Hinglish (Apex / Swift): three notebooks per variant, meant to run in this order:
hinglish_apex_alignment_heads.ipynb / hinglish_swift_alignment_heads.ipynb — discovers the DTW alignment heads these fine-tuned checkpoints need for real per-word timestamps (they're not official OpenAI/distil-whisper releases, so they aren't in sherpa-onnx's pre-baked table). Already run once; the discovered heads are documented in the notebook and already baked into step 3.
To reproduce for a different fine-tuned Whisper checkpoint entirely (any language,
any domain), the same three-step pattern applies: convert to OpenAI format, discover
alignment heads for that specific checkpoint, then export with attention using those
heads.
Getting a Hugging Face token to run the notebook
AI4Bharat's source model repos are "gated" — free to use, but you need to accept
their terms once per language before downloading. To run ai4bharat_export_pipeline.ipynb
yourself:
Create a free account at huggingface.co if you don't
have one.
Visit https://huggingface.co/ai4bharat/indicconformer_stt_<lang>_hybrid_ctc_rnnt_large
for whichever language you want (e.g. hi, ta, bn), log in, and click
"Agree and access repository". Do this once per language you plan to convert.
In Google Colab: click the key icon (🔑) in the left sidebar → Add new secret
→ name it exactly HF_TOKEN → paste your token as the value → enable
Notebook access.
Never paste the token value directly into a code cell — the notebook reads it
securely via userdata.get('HF_TOKEN').
That's all the setup needed before running the notebook top to bottom.
Credit and license
The 10 Indic language models are built on the work of
AI4Bharat (IIT Madras) — all underlying model
weights, architecture, and training are theirs. This repo only re-packages their
published checkpoints into a smaller, quantized, masked ONNX format for offline
mobile deployment.
The English model is built on
NVIDIA NeMo's fast-conformer CTC architecture and
checkpoint, converted to ONNX and published by the
sherpa-onnx / k2-fsa project as part of
their open pretrained-model collection. This repo only applies additional int8
quantization on top of their existing ONNX export.
The Hinglish models (Apex / Swift) are built on
Oriserve's Whisper-Hindi2Hinglish-Apex and
Whisper-Hindi2Hinglish-Swift fine-tunes, which are themselves built on
OpenAI Whisper (Large-V3 and base respectively).
This repo only converts Oriserve's published checkpoints to ONNX and applies int8
quantization for offline mobile deployment — all fine-tuning, training data curation,
and the underlying Whisper architecture/pretraining are theirs and OpenAI's
respectively.
Full credit for the actual speech recognition research and training belongs to
AI4Bharat, NVIDIA NeMo, OpenAI, Oriserve, and the sherpa-onnx/k2-fsa team
respectively. Released under Apache 2.0, matching the source models' license.
Contributing
If you convert additional AI4Bharat languages using this same pipeline and want to
add them here, open a discussion on this repo.
