indic-transliterate (prototype)
Roman/Hinglish/code-mixed → native Indic script transliteration, built to sit
in front of a TTS pipeline (pronunciation-preserving transliteration, not
translation — e.g. "EMI" → "ईएमआई", not "किस्त").
⚠️ Status: early prototype, not production-ready
This checkpoint was trained on a deliberately small, capped slice of data
(3,000 word pairs per language from AI4Bharat Aksharantar, ~13k examples
total after language-temperature sampling and customer-support-data mixing)
for 3 epochs, as an end-to-end pipeline validation step — not a
production training run.
Measured on a held-out test set:
| Metric | Value |
|---|
| Character Error Rate | 91% |
| Exact match | 0% |
| Script validity | 99.9% |
Script validity (99.9%) shows the model reliably learned which script to
emit per language tag — the architecture and data pipeline are wired
correctly. Character accuracy (91% CER) is not usable yet — that needs a
full-scale training run (full per-language data volume, more epochs) before
this should be used for anything beyond pipeline testing.
What this is (and isn't)
- Is: a from-scratch, ~7.7M-parameter character-level Transformer
encoder-decoder (
torch.nn.Transformer-based), with <2xx> language-tag
tokens (same convention as IndicXlit/NLLB/mBART), trained via Hugging Face
transformers.
- Isn't: AI4Bharat's IndicXlit checkpoint or its fairseq architecture.
IndicXlit's fairseq codebase is not compatible with Python 3.12 (mutable
dataclass-default fields fairseq declares are hard-rejected by Python
3.11+'s
dataclasses module, across multiple files, not a single
patchable spot) — see the training repository's README for the full
writeup. This model uses the same task framing and tagging convention on a
maintained stack instead of vendoring IndicXlit's literal weights.
Training data
AI4Bharat Aksharantar
— word-level Roman↔native pairs, 21 languages. Aksharantar's license is
mixed (CC-BY for manually-collected rows, CC0 for mined rows); this model
card carries
cc-by-4.0 as the safe upper bound. See the training repo's
README for the full licensing note, including why Google Dakshina (CC BY-SA,
share-alike) was
not used for this checkpoint.
Usage
1from transliteration.inference import TransliterationEngine
2
3engine = TransliterationEngine.from_checkpoint("path/to/this/checkpoint")
4engine.transliterate("mera emi pending hai", language="hi")
Full pipeline code (data download/validation/training/evaluation/inference/
TTS integration): see the accompanying repository files.
Intended use
Pipeline validation and further fine-tuning. Not intended for production
transliteration until retrained at full data scale — see Known Limitations
in the training repo's README.
