Uzbek OCR correction model for old books. It is built to take noisy scanned book text, usually in Cyrillic, and turn it into cleaner Uzbek Latin text.

Authors

Sardor Islomov — lead author
Davron Ibrokhimov

This model is focused on a specific problem: many older Uzbek books survive as scans of printed Cyrillic books, and raw OCR from those scans is often hard to read, hard to search, and hard to reuse. Since Latin is the official script used in Uzbekistan today, this model is intended to help turn damaged OCR output into more usable Latin Uzbek text and increase accessible Latin book content.

This is the OCR-specialized variant of the rubai-corrector family.

rubai-corrector-base: general fine-tuning base
rubai-corrector-transcript-uz: ASR transcript display cleanup
rubai-corrector-ocr-books-uz (this model): old-book OCR correction

What It Is Good At

Use this model when your input looks like OCR from scanned Uzbek books, especially older Cyrillic books.

It is good at:

Cyrillic Uzbek -> Latin Uzbek conversion
OCR letter damage repair
apostrophe and quote normalization
fixing split or merged words caused by OCR
restoring cleaner punctuation and hyphenation
preserving multiline text reasonably well for poetry and book passages

Typical OCR issues it can repair:

савод чнқаришга -> savod chiqarishga
кйлдан -> qo'ldan
козон-позон -> qozon-pozon
суриштнриб -> surishtirib

It is not the right model for:

ASR transcript display polishing
Russian recovery from Latin ASR text
generic denormalization tasks outside OCR books

Base And Data

Base model: rubai-corrector-base
Fine-tuned on a curated OCR correction dataset built from scanned Uzbek books
Fine-tuning train split size: 10,730 pairs
Full prepared dataset size: 12,642 pairs from 54 books
Each training pair maps noisy OCR text to corrected Uzbek Latin text
The dataset includes multiple OCR views of sampled book content and a normalized Latin target style
Targets were standardized to keep apostrophes, quotes, and dashes consistent

Quick Start

python

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_path = "islomov/rubai-corrector-ocr-books-uz"

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSeq2SeqLM.from_pretrained(model_path)

text = "Бошим айланиб, кўз олдим айланади. Ана шаҳар, ҳамма қўлида алифбе китоби, савод чнқаришга туртинмоқдалар."
inputs = tokenizer(f"correct: {text}", return_tensors="pt")
output_ids = model.generate(**inputs, max_new_tokens=512)
prediction = tokenizer.batch_decode(output_ids, skip_special_tokens=True)[0]
print(prediction)

Expected output:

Boshim aylanib, ko'z oldim aylanadi. Ana shahar, hamma qo'lida alifbe kitobi, savod chiqarishga turtinmoqdalar.

The model expects the correct: prefix at inference time.

Real Example Outputs

These examples are taken from the actual held-out OCR evaluation for this checkpoint.

OCR damage: broken letters

text

Input:     Бошим айланиб, кўз олдим айланади. Ана шаҳар, ҳамма қўлида алифбе китоби, савод чнқаришга туртинмоқдалар.
Output:    Boshim aylanib, ko'z oldim aylanadi. Ana shahar, hamma qo'lida alifbe kitobi, savod chiqarishga turtinmoqdalar.

Corrects:

чнқаришга -> chiqarishga
full Cyrillic -> Latin conversion
apostrophe restoration in ko'z, qo'l

OCR damage: broken words

text

Input:     уриб турмаса кўнгли жойига тушмас, каерга борса шундан сўз очишни, кимни кўрмасин шу хакда бирон янги гап эшитишни, кандай мажлнс булмасин шу тўгрида масала кўйилишини хохлар ва ўнг келганда фурсатни кйлдан бермас эди.
Output:    urib turmasa ko'ngli joyiga tushmas, qayerga borsa shundan so'z ochishni, kimni ko'rmasin shu haqda biron yangi gap eshitishni, qanday majlis bo'lmasin shu to'g'rida masala qo'yilishini xohlar va o'ng kelganda fursatni qo'ldan bermas edi.

Corrects:

каерга -> qayerga
мажлнс -> majlis
кйлдан -> qo'ldan
apostrophes across the line
multiple OCR-damaged words in one sentence

OCR damage: mixed lexical corruption

text

Input:     "Сотасанми?", дедим, "Сотаман", деди. Пул билан савдо келишмагандан кейин, ош билан айирбош килмокчи бўлди. Кўндим. Ошни козон-позон, ўчок дамгирлари билан шу унга алишдим.
Output:    "Sotasanmi?", dedim, "Sotaman", dedi. Pul bilan savdo kelishmagandan keyin, osh bilan ayirbosh qilmoqchi bo'ldi. Ko'ndim. Oshni qozon-pozon, o'choq damgirlari bilan shu unga alishdim.

Corrects:

килмокчи -> qilmoqchi
козон-позон -> qozon-pozon
ўчок -> o'choq
quote and apostrophe normalization

OCR damage: names and rare words

text

Input:     Малика овозаларнинг қанчалик ростлигини Бобир саройида хизматда юрган невараси Баҳодирдан суриштнриб билмоҳчи бўлди.
Output:    Malika ovozalarning qanchalik rostligini Bobir saroyida xizmatda yurgan nevarasi Bahodirdan surishtirib bilmoqchi bo'ldi.

Corrects:

суриштнриб -> surishtirib
named entities and historical names
full Cyrillic -> Latin conversion without losing meaning

OCR damage: poetry and layout

text

Input:     Айт-чн, йигит, Кимни ўйладинг, Куйлаганинг ким эди, Айт, ким? Ким ўртади ҳажрида сени, Кимга орзунг бўлмоқ муяссар?
Output:    Ayt-chi, yigit,
           Kimni o'ylading,
           Kuylaganing kim edi,
           Ayt, kim?
           Kim o'rtadi hajrida seni,
           Kimga orzung bo'lmoq muyassar?

Corrects:

Айт-чн -> Ayt-chi
line segmentation for verse-like text
apostrophes in o'ylading, o'rtadi

Cyrillic book text -> clean Latin

text

Input:     Фронтдан эсиб турган ел охирги мартаба ўлароқ ўзи билан бомба, шрапнель жаҳаннам машиналари ва оғир тўплардан портлаган порохларнинг заҳар тутунларини улаштириб, Оқ денгизнинг зилол қўйнига тўкди.
Output:    Frontdan esib turgan yel oxirgi martaba o'laroq o'zi bilan bomba, shrapnel jahannam mashinalari va og'ir to'plardan portlagan poroxlarning zahar tutunlarini ulashtirib, Oq dengizning zilol qo'yniga to'kdi.

Corrects:

long continuous Cyrillic prose
apostrophes in o'laroq, og'ir, qo'yniga, to'kdi
punctuation and readability preserved across a long sentence

Challenging example: many OCR fixes in one long passage

text

Input:     Уни хар кандай от хам бир козон кайновича муддатдан ортик кўтариб юролмас, унга савашда тенг келгувчи топилмас эди, ана шу ботирлар ботирини охири Жалолиддинга рўбарў килди. "Дев" ўз одатича чинкириб ракибига ташланди, аммо дам ўтмай овози ўчди.
Output:    Uni har qanday ot ham bir qozon qaynovicha muddatdan ortiq ko'tarib yurolmas, unga savashda teng kelguvchi topilmas edi, ana shu botirlar botirini oxiri Jaloliddinga ro'baro' qildi. "Dev" o'z odaticha chinqirib raqibiga tashlandi, ammo dam o'tmay ovozi o'chdi.

Corrects:

хар -> har
кандай -> qanday
козон -> qozon
орт ик -> ortiq
чинкириб -> chinqirib
ракибига -> raqibiga
long multi-clause OCR text with quotes and apostrophes preserved

Why This Model Matters

For old Uzbek books, the hard part is often not just script conversion. It is OCR damage correction:

dropped or corrupted letters
broken apostrophes
bad word boundaries
damaged names
noisy punctuation
messy multiline book text

This model is designed for that exact problem.

Practical Limits

It still struggles more on:

title pages
bibliographic blocks
repeated headers and page furniture
glossary and footnote-heavy lines
extremely damaged OCR with severe spacing noise

For those cases, manual review is still recommended.

Files

test_model.py: runnable local example script

Acknowledgements

Special thanks to Davron Ibrokhimov for sponsoring this work and making it possible to keep these models open.

Thank you to the community that supports Uzbek language technology. In particular:

MetaSell for support and resources
Kotib for their support and collaboration on Uzbek STT
Global Move for backing open Uzbek NLP work

Thanks to Arofat, Gulimshaxnoz, and many others who contributed in ways big and small. The list is too long to fit here, but every contribution matters and is appreciated.

Support my works and open-source movement: https://tirikchilik.uz/islomovs