Telugu OCR — line recognition, trained from scratch
A Telugu text-line recogniser built end to end for Telugu: a convolutional-stem CTC image
encoder joined to a Telugu grapheme language model by cross-attention. Give it an image of
one line of Telugu text and it returns the text.
It makes about one fifth as many errors as the best general-purpose OCR engine available
for Telugu.
Results
Scored on harsha-desaraju/telugu-line-ocr-bench
— 1044 line crops whose transcriptions a human checked against the image. Every engine read
the identical images and was scored against the identical references. All figures are
percentages; lower is better except the last column.
engine
CER
AER
WER
exact line
this model (joint decode)
1.19
1.98
9.53
71.7
this model (ctc decode)
1.34
2.23
10.52
69.7
Tesseract 5.5 (tel, psm 13)
6.64
9.99
31.98
26.7
Surya 1 (0.14.x)
9.88
13.54
36.10
24.1
PaddleOCR 3.x (te)
9.21
13.84
42.14
17.2
AER — akshara error rate — is the number to read for Telugu. A Telugu akshara is usually
three or four code points, so CER charges a missing vowel sign one edit and a garbled
cluster two or three, while a reader perceives one wrong syllable either way.
By AER this model is 5.0× better than Tesseract, 6.8× better than Surya and 7.0×
better than PaddleOCR. It transcribes 71.7% of lines with zero errors, against 26.7%
for the strongest baseline — the difference between output you can read and output you have
to correct.
Its remaining errors: 244 substitutions, 78 insertions, 205 deletions across 26,650
reference aksharas. Deletions dominate, mostly dropped spaces.
Usage
The architecture is custom, so the model ships its own code and needs
trust_remote_code=True. Nothing else to install beyond transformers, torch, pillow
and regex.
python
1from PIL import Image
2from transformers import AutoModel, AutoTokenizer
34model = AutoModel.from_pretrained("harsha-desaraju/telugu-ocr-model",5 trust_remote_code=True).eval()6tokenizer = AutoTokenizer.from_pretrained("harsha-desaraju/telugu-ocr-model",7 trust_remote_code=True)89# one image -> str10print(model.transcribe(Image.open("line.png"), tokenizer))1112# a list of images -> list[str]13print(model.transcribe([Image.open(p)for p in paths], tokenizer))1415# fast mode: no beam search, ~9x quicker for ~11% more error16print(model.transcribe(images, tokenizer, decode="ctc"))
On a GPU, move the model first: model = model.to("cuda").
Input requirements
Feed it one line of text per image. Preprocessing is handled internally — grayscale,
resize to height 64 preserving aspect ratio, pad the width to a multiple of 8 — so pass a
plain PIL.Image. A line wider than 2048px once scaled to height 64 cannot be encoded and
returns "" rather than being silently truncated.
There is no page layout analysis or line detection here. Segment lines yourself first.
Decode modes
The same weights support three decoding strategies, selected per call.
mode
how
AER
when to use
ctc
per-frame argmax off the CTC head, collapse repeats, drop blanks
2.23
bulk throughput — ~9× faster, only ~11% worse
beam
beam search over the text decoder
—
language-model reading alone
joint
beam n-best rescored by λ·logP_ctc + (1−λ)·logP_attn, λ=0.3
1.98
best accuracy (default)
joint is the headline number and the default. Reach for ctc when labelling millions of
crops: it costs ~11% more AER and runs about nine times faster, with no beam search at all.
Architecture
~81M parameters, float32.
line image (1×64×W, W ≤ 2048)
↓ conv stem: 6 blocks, 32→64→128→256→320→384, height→1, width÷8
↓ 10-layer pre-LN transformer encoder, d=384, 8 heads, MLP 1536
├──→ CTC head (2049 classes = 2048 aksharas + blank)
↓ enc_to_dec: linear 384 → 512
↓ 16-layer GPT decoder, d=512, SwiGLU 1368, 8 heads, ctx 256
↓ cross-attention on every second block, zero-init tanh gate
→ text
Counts are over the checkpoint's 388 tensors, so they include the decoder's non-trainable
positional table.
Two design choices carry most of the quality:
Grapheme (akshara) tokenisation. Text is split with regex.\X into Unicode grapheme
clusters — one cluster, one token, no BPE. Vocabulary 2048. A Telugu syllable is a single
token, so the decoder predicts syllables rather than assembling them from fragments of code
points.
Two heads over one encoder. The CTC head sees the image directly and cannot hallucinate
text that is not there; the attention decoder carries language knowledge and repairs what
CTC garbles. Keeping both allows rescoring one against the other, which is where the best
numbers come from.
Training
Three pretrained pieces, then two fine-tuning stages:
Image encoder — trained with CTC loss on Telugu line images.
Text decoder — a 16-layer causal grapheme language model pretrained on Telugu text.
Stage 1 — both backbones frozen; only the new cross-attention, its norms, the gates
and the bridge are trained. The gate is zero-initialised behind a tanh, so the image
contributes nothing at initialisation and is phased in as the gate learns.
Stage 2 — everything unfrozen and trained end to end, with the encoder's CTC head
kept as an auxiliary term so it cannot drift away from the features the decoder relies
on.
Training data was synthetic Telugu line renders (many fonts and sizes, degraded with a
scan-simulation pipeline) mixed with real line crops from Telugu Wikisource proofread page
scans. Real crops were not augmented — they are already degraded.
Limitations
Printed book text only. Trained on printed Telugu from book scans and synthetic
renders. No handwriting, no signage, no scene text, no historical orthography.
Single lines only. No layout analysis or line detection; segment before calling.
Hard width ceiling of 2048px at height 64. Longer lines must be split.
Citation
bibtex
1@misc{telugu_ocr_model,
2 title = {Telugu OCR: a grapheme-level CTC encoder with a cross-attending Telugu language model},
3 author = {Desaraju, Harsha},
4 year = {2026},
5 url = {https://huggingface.co/harsha-desaraju/telugu-ocr-model}
6}
