Tara: Frontier Hindi Transcription Model
Watch the launch video
here. Try via API
here.
Tara is a frontier automatic speech-recognition model for Hindi and mixed-code (Hinglish)
transcription. On the AI4Bharat Vistaar Hindi benchmark suite it achieves state-of-the-art
aggregate accuracy, outperforming leading commercial Hindi ASR systems on the 7-benchmark
Vistaar mean, while natively handling Hindi–English code-switched speech through a dedicated mixed-code
mode that renders English words in Latin script and Hindi in Devanagari, the way real
Hinglish is written.
Highlights
- State-of-the-art Vistaar Hindi aggregate: 12.06 WER mean over the 7 Vistaar sets, ahead
of Sarvam Saaras-v3 (12.32), with wins on Kathbath, GramVaani, IndicTTS and CommonVoice-hi.
- Native code-switching: 8.37 WER on Code-Switch FLEURS (CS-FLEURS) Hindi–English read code-switch via Tara's
mixed-code mode, competitive with the best commercial systems.
- Robust across domains: read speech, noisy speech, telephony (GramVaani 21.03 vs Sarvam
23.00), spontaneous conversation (IndicVoices), and accented adult/child speech (HiACC).
- Bilingual: retains strong English (6.68 WER CommonVoice-en, 4.55 FLEURS-en).
- Standard tooling: loads with 🤗 Transformers exactly like
openai/whisper-large-v3.
Usage
1import librosa
2import torch
3from transformers import WhisperProcessor, WhisperForConditionalGeneration
4
5repo = "Trelis/tara"
6processor = WhisperProcessor.from_pretrained(repo)
7model = WhisperForConditionalGeneration.from_pretrained(
8 repo, torch_dtype=torch.bfloat16).to("cuda")
9
10tk = processor.tokenizer
11hi, en, mc = (tk.convert_tokens_to_ids(t) for t in ("<|hi|>", "<|en|>", "<|mixedcode|>"))
12trn, nts = (tk.convert_tokens_to_ids(t) for t in ("<|transcribe|>", "<|notimestamps|>"))
13
14audio_16k, _ = librosa.load("clip.wav", sr=16000, mono=True)
15feats = processor(audio_16k, sampling_rate=16000,
16 return_tensors="pt").input_features.to("cuda", torch.bfloat16)
17
18# Example 1: pure Hindi
19out = model.generate(input_features=feats,
20 forced_decoder_ids=[(1, hi), (2, trn), (3, nts)],
21 max_new_tokens=444)
22print(tk.decode(out[0], skip_special_tokens=True))
23
24# Example 2: Hindi-English mixed-code, inject <|mixedcode|> right after the language token.
25# Language auto-detection also works: generate one step unforced and the FIRST generated
26# token is the language token; then inject <|mixedcode|> after it and continue.
27out = model.generate(input_features=feats,
28 forced_decoder_ids=[(1, hi), (2, mc), (3, trn), (4, nts)],
29 max_new_tokens=444)
30print(tk.decode(out[0], skip_special_tokens=True))
The mixed-code mode (the <|mixedcode|> prefix above) conditions generation only: on pure-Hindi
audio it neither degrades accuracy nor forces transliteration; on mixed-code audio it renders
English words in Latin script.
Evaluation
Evaluation code, the exact text normalizer, and Tara's per-utterance predictions for every
benchmark below are published at
TrelisResearch/tara,
so all numbers can be reproduced or re-scored under alternative normalizers.
Protocol. All numbers are corpus WER after light text normalization* (Unicode NFC plus
punctuation removal; nukta and all vowel and nasal marks preserved). All systems are scored
on clips ≤ 30 s with identical references. Commercial-system results are measured by us
under the same protocol; they are not vendor-reported figures.
Vistaar Hindi benchmark (WER ↓)
IndicVoices-500 is a 500-sample spontaneous-speech control from the IndicVoices validation
split; it is not part of the Vistaar mean.
Code-switching (Hinglish) benchmarks (WER ↓)
Tara and Sarvam are measured in their code-mixed modes.
English (WER ↓)
Scored with the standard Whisper English normalizer.
* Normalization: unicodedata.normalize("NFC"), lowercasing, then removal of punctuation
and symbols (। , . ? ! " : ; - – — “ ” ( ) [ ] < > / ~ % ₹ $ …), invisible formatting
characters (zero-width joiner/space) and the Unicode replacement character; apostrophes are
kept. The ≤30 s rule excludes 2 clips on GramVaani, 2 on IndicTTS and 1 on FLEURS-hi; no
other set has any. Measurement error is small: re-runs across hardware and precision agree
to within 0.1 WER. There is also slight noise in the reference labels (for example
inconsistent nukta spelling: both हज़ार and हजार appear as references within GramVaani, and
both ज़्यादा and ज्यादा within MUCS), but this should not affect the numbers by much.
Limitations
- Mode selection: peak accuracy comes from picking the mode per clip (Hindi, mixed-code, or
English). When the language mix is unknown, the Hindi mixed-code mode is a safe default: on
pure-Hindi audio it produces pure-Hindi transcripts with no measured accuracy loss, and on mixed
audio it handles the code-switching. Automatic language detection is also supported: let the
model generate the language token and inject mixed-code after it (see Usage).
- Clip length: evaluated on clips ≤ 30 s; longer audio should be chunked (standard Whisper
practice).
- Hindi–English only; other Indic languages are out of scope for this release.
Intended use
Transcription of Hindi and Hindi–English code-switched speech: voice assistants, contact-center
analytics, media captioning, and speech data pipelines.
Model details
- Architecture: Whisper large-v3 (encoder–decoder, 1.55B params) + mixed-code mode
- Languages: Hindi (hi), English (en), Hindi–English code-switch
- Sample rate: 16 kHz input
- I/O: ≤30 s audio per window → text
- License: Apache 2.0
Attribution
We thank
Gram Vaani for permission to use the
Gram Vaani ASR Challenge 2022 Corpus in training Tara. Gram Vaani
builds community-anchored voice based engagement platforms ('Mobile Vaani' clubs) that give underserved and
marginalised communities a channel to access information and express themselves.
License
This model is released under the Apache License 2.0.
Citation
If you use Tara in your work, please cite:
1@misc{trelis2026tara,
2 title = {Tara: Frontier Hindi Transcription Model},
3 author = {{Trelis Research}},
4 year = {2026},
5 url = {https://huggingface.co/Trelis/tara}
6}
