Hinglish → Devanagari Transliteration Model (v3)
15M-param character-level seq2seq Transformer for romanized Hindi (Hinglish)
→ Devanagari transliteration, built for a Chinese-smart-pinyin-style Hindi IME.
Trained on
IB-Emper/hinglish-translit-v2
(548,943 pairs) plus the Dakshina lexicon and a capped Aksharantar split.
Files
best.pt — fp32 checkpoint, ~60MBbest_int8.pt — dynamically INT8-quantized checkpoint (feedforward layers only;
attention projections stay fp32 — PyTorch's dynamic quantization does not touch
nn.MultiheadAttention projections), ~37MBmodel.py — architecture + tokenizer + checkpoint load/save (needed to load either file)
Architecture
d_model=384, nhead=6, 4 encoder + 4 decoder layers, dim_feedforward=1280,
dropout=0.1, max_len=64, norm_first (pre-LN), weight-tied embedding/output.
Char-level tokenizer, frozen vocab stored in the checkpoint. 15.13M params.
Usage
1from model import load_checkpoint, BOS, EOS
2import torch
3
4model, tok, extra = load_checkpoint("best.pt", map_location="cpu")
5# beam search / decoding left to the caller -- see the project repo for a
6# reference implementation (KV-cached beam search decoder).
Loading best_int8.pt requires reconstructing the model, applying
torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8),
then calling load_state_dict on the result — the packed quantized weights
won't load directly into a plain fp32 model.
Eval (Dakshina lexicon, Hindi test set, beam+LM rescoring)
| acc@1 | acc@3 |
|---|
| ALL | 62.5% | 77.0% |
| FREQ tier (own corpus-frequency tiering) | 84.2% | — |
For comparison, IndicXlit (AI4Bharat/Microsoft, 11M params split across 21
languages) reports 60.5% acc@1 / 81.9% acc@3 unreranked, 72.4% acc@1 with
their own reranking, on the same official Dakshina test set. This model is
competitive on acc@1, not superior — it loses to IndicXlit's reranked top-1
and to their top-3 even before reranking. It was built and tuned for casual/
code-switched chat-style input specifically, which IndicXlit was not trained
or evaluated on; that comparison has not been run head-to-head.
Known limitations
- Confusable near-homophone pairs (मेरा/मारा, क्या/किया-type errors) may or may not always have a correct prediction.
- ~13% of training pairs retain inline Latin code-switch words on the
Devanagari side by design (realistic chat register), not a bug.
- Not yet quantized beyond dynamic INT8;
