Su-śrotā — Sanskrit ASR (finetuned IndicConformer-CTC)
A Sanskrit speech-recognition model finetuned for
śāstric and recitational Sanskrit (chant and
prose), where general-purpose Sanskrit ASR degrades badly. It is the model behind
Vāgbodhinī and
Su-śrotā, live Sanskrit
chant-practice and dictation tools.
- Base: AI4Bharat IndicConformer (
EncDecHybridRNNTCTCBPEModel, ~129 M params). We use the
CTC head on the Sanskrit token slice.
- Author: Prof. Prathosh A P, Indian Institute of Science, Bengaluru.
Checkpoints in this repo
| File | Description |
|---|
sushrota_sanskrit_asr_v13b.nemo | Current / deployed. Adds consented real-world ("flywheel") data on top of the curated base — far more robust on in-the-wild user recordings. |
sushrota_sanskrit_asr_v5.nemo | Prior version, curated data only. Retained for reproducibility. |
Results
The headline gain of the current model is real-world robustness. Evaluated on a leakage-free,
327-clip held-out set of in-the-wild user recordings (phones, rooms, varied speakers):
| eval set | metric | v5 (curated only) | v13b (current) |
|---|
| in-the-wild user audio | CER | 7.70% | 4.36% (−43%) |
| in-the-wild user audio | WER | 45.4% | 30.4% |
| in-the-wild user audio | SN-WER | 20.7–24.4% | 10.8–13.2% |
On curated/studio domains, v13b preserves the base model's quality while adding that robustness:
| domain | CER | WER | SN-WER (sandhi-normalised) |
|---|
| Studio held-out (305 clips) | 4.4% | 20.2% | 13.0–15.7% |
| Bhāgavata chant (968) | ~6.0% | ~46% | ~22–26% |
| Vedānta prose (718) | ~7.2% | ~31% | ~15–19% |
Note on WER vs SN-WER: Sanskrit word boundaries are largely orthographic (sandhi fuses words),
so raw WER over-penalises boundary disagreements. Roughly half of the WER is spacing; CER and
SN-WER (sandhi-normalised) are the meaningful numbers.
Training Data
The deployed checkpoint (v13b) was trained on 17.4 hours / 6,438 utterances, combining curated
scholar recordings with consented, quality-tiered in-the-wild data.
| Source | Utterances | Hours | Description |
|---|
| Scholar recordings (force-aligned) | 2,139 | 6.06 | 21 reciters reading Bhāgavata Purāṇa, Upaniṣad, and stotra texts; forced-aligned to reference |
| Recitation corpus | 2,504 | 7.59 | Pre-segmented Upaniṣad (3.0 h), Gītā / Ṛgveda (3.0 h), and additional recitation (1.6 h) |
| TTS-speaker augmentation | 637 | 2.00 | Studio/synthetic voices for speaker robustness |
| Consented flywheel (in-the-wild) | 1,158 | 1.73 | User recordings from the practice tools, clean tier, ~11 input scripts |
| Total | 6,438 | 17.37 | |
Split: ~90% curated/scholar (15.65 h) + ~10% consented in-the-wild (1.73 h).
Data pipeline & quality tiering
In-the-wild audio is collected with consent through the practice tools (users read a known
reference text) and automatically quality-graded at the akṣara level against that reference. Each
clip is tiered — pass / override (clean, trainable), review (partial match), or low /
unclear (archived). Only the clean tiers enter training.
The auto-grader was audited for reliability: all quarantined review clips were re-decoded with
the improved model and promoted back to gold only when they scored a perfect akṣara match. Just
4.3% proved to be model error — confirming the remaining ~96% are genuine reader deviations
(mispronunciations, disfluencies, noise), i.e. the grader correctly quarantines real problems rather
than mere model disagreements.
Consented corpus
Beyond what is trained on, the flywheel has banked ~28 hours of consented Sanskrit audio from
~1,500 speakers across 11 input scripts (Devanāgarī, Kannada, Telugu, IAST, and others) — a growing
resource for future, more speaker- and script-diverse training rounds.
Usage
1import json, numpy as np, torch, soundfile as sf
2import nemo.collections.asr as na
3
4M = na.models.EncDecHybridRNNTCTCBPEModel.restore_from("sushrota_sanskrit_asr_v13b.nemo").eval()
5
6OFF, V, BLANK = 4096, 256, 5632 # Sanskrit token slice of the aggregate vocab
7def greedy(wav): # wav: 16 kHz mono float32
8 sig = torch.tensor(wav).unsqueeze(0); sl = torch.tensor([len(wav)])
9 with torch.no_grad():
10 enc, _ = M.forward(input_signal=sig, input_signal_length=sl)
11 lp = M.ctc_decoder(encoder_output=enc)[0].cpu().numpy()
12 cols = [BLANK] + list(range(OFF, OFF + V))
13 P = lp[:, cols]; P = P - (P.max(1, keepdims=True) +
14 np.log(np.exp(P - P.max(1, keepdims=True)).sum(1, keepdims=True))) # re-log_softmax on slice
15 ids = P.argmax(1)
16 sub = M.tokenizer.tokenizers_dict["sa"] # Sanskrit SentencePiece
17 out, prev = [], -1
18 for i in ids:
19 i = int(i)
20 if i != prev and i != 0: out.append(sub.ids_to_tokens([i - 1])[0])
21 prev = i
22 return "".join(out).replace("▁", " ").strip()
23
24wav, sr = sf.read("clip.wav", dtype="float32")
25print(greedy(wav))
The model is an aggregate multilingual IndicConformer; decode on the Sanskrit slice as above.
Provenance & consent
Scholar recordings were contributed for the purpose of building this model. In-the-wild clips are
collected only with explicit user consent and store no raw IP or personal identifiers (an anonymous
per-session id only). Individual reciter names are not published.
Related
- Vāgbodhinī (chant-practice tool + full experiment report): https://github.com/prathoshap/sushrota-sanskrit-asr
- Vāgdhenu (metre-aware Sanskrit chant TTS):
prathoshap/vagdhenu
License
Finetuned from AI4Bharat's IndicConformer — please observe the base model's license terms.
Citation
Prathosh A P, Su-śrotā: Scholar-grade Sanskrit ASR and metre-aware chant practice, Indian
Institute of Science, Bengaluru, 2026.
