trocr-large-rukopys-hw
A
TrOCR-Large (
VisionEncoderDecoderModel, ~558M params) line recognizer for
handwritten Ukrainian document text, fine-tuned on the
Rukopys dataset.
The training data included handwritten, printed, annotation and table region crops from Rukopys.
TL;DR
| value |
|---|
| Architecture | TrOCR-Large (ViT-L/16 encoder @ 384px + TrOCR decoder), VisionEncoderDecoderModel |
| Parameters | ~558M |
| Init from | Hukyl/trocr-large-uk-handwritten-real (real-only consolidation) |
| Tokenizer | extended Cyrillic vocab, size 50336 (ships with this repo) |
| Handles | handwritten, printed, table, annotation |
| Gold-val CER / WER | 0.0825 / 0.2583 |
handwritten-class CER | 0.0750 |
| Input | a single cropped text region/line (RGB) |
| Output | the transcribed string (metric-normalized character set) |
Intended use
Recognizing cropped line/region images from handwritten Ukrainian documents, downstream
of a layout detector. This recognizer was tuned for the strongest handwritten class reading.
For a recognizer more balanced across all region classes, see sibling model
Hukyl/trocr-large-rukopys.
How to use
Note: load the processor from this repo, not from microsoft/trocr-large-handwritten —
the tokenizer here is an extended Cyrillic vocabulary (50336 vs stock 50265).
1from PIL import Image
2from transformers import VisionEncoderDecoderModel, TrOCRProcessor
3
4repo = "Hukyl/trocr-large-rukopys-hw"
5processor = TrOCRProcessor.from_pretrained(repo) # load from THIS repo
6model = VisionEncoderDecoderModel.from_pretrained(repo).eval()
7
8crop = Image.open("region_crop.png").convert("RGB") # one detected line/region
9pixel_values = processor(images=crop, return_tensors="pt").pixel_values
10generated = model.generate(pixel_values, max_new_tokens=256, num_beams=1) # greedy
11text = processor.batch_decode(generated, skip_special_tokens=True)[0]
12print(text)
Lineage & training curriculum
Rukopys gold-only fine-tuning from a real-only consolidation checkpoint, with no
silver/pseudo-labels. Full fine-tune, optimizer: AdamW via the 🤗 Seq2SeqTrainer,
cosine LR decay, max_target_length 256, online augmentation, seed 42, single L40S.
- Base —
Kansallisarkisto/cyrillic-htr-model
(Apache-2.0): microsoft/trocr-large-handwritten fine-tuned on ~30k historical
Cyrillic rows by the Finnish National Archives; extended tokenizer (vocab 50336).
- Mixed pretrain —
Hukyl/trocr-large-uk-handwritten
(6 ep, LR 4e-5): UkrHandwritten ×3 + Cyrillic Handwriting +
part of pumb-ai/synthetic-cyrillic-large.
- Real-only consolidation —
Hukyl/trocr-large-uk-handwritten-real
(16 ep, LR 2e-5): UkrHandwritten ×2 + Cyrillic Handwriting, no synthetic data.
The general Ukrainian handwriting reader.
- Rukopys gold fine-tune (20 ep, LR 1e-5, warmup 0.05, weight decay 0.01): the
human-labeled gold training data, with page-level validation split.
Note: a smaller validation split was used to give more of the Rukopys gold split
to training: 21,720 train / 1,223 val crops. The published weights are the
best-val-CER epoch (10 of 20).
Hyperparameters (as launched)
| hyperparameter | value |
|---|
| epochs | 20 |
| batch size | 32 |
| learning rate | 1e-5 |
| warmup ratio | 0.05 |
| weight decay | 0.01 |
| max target length | 256 tokens |
| optimizer / schedule | AdamW, cosine decay |
| online augmentation | on (per-region profile) |
| eval | greedy (num_beams=1), full val each epoch |
| precision / device | bf16, CUDA (single L40S) |
| seed | 42 |
Fine-tune ran 13,300 steps in ~2.5h.
Online data augmentation
Online augmentation was applied during training. Only training crops were augmented (no augmented validation was measured).
Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.
| region type | geometric (one) | photometric (one or two) |
|---|
| handwritten, annotation | margin pad 2–15% / trim 1–5%, rotation ±1–5°, elastic distortion (α=25, σ=5), baseline warp (amp 2–8 px, freq 0.5–2.0) | paper-colour shift (LAB a±10 / b±15), Gaussian noise (σ 5–15), JPEG recompression (q 30–65), contrast/gamma (0.7–1.3 / 0.6–1.5), morphological erode/dilate (kernel 2) |
| printed, table | margin pad 2–15% / trim 1–5%, rotation ±1–3° | paper-colour shift (a±5 / b±10), Gaussian noise (σ 3–10), JPEG recompression (q 40–70), contrast/gamma (0.8–1.2 / 0.8–1.3), morphological erode/dilate (kernel 2) |
These simulate scanner/paper variation, ink thinning/bleed, and natural handwriting
deformation, widening the range of appearances beyond the raw training crops.
Results
Protocol: greedy decode, max_new_tokens=512, page-level gold validation split,
1000-sample draw, seed 42. Labels are metric-normalized (LaTeX → Unicode, table PSV
canonicalized) so CER/WER reflect the scored character set.
Overall
| metric | value |
|---|
| CER | 0.0825 |
| WER | 0.2583 |
| exact-match accuracy | 0.4280 |
| n_samples | 1000 |
A full-val (1,223-crop) training eval pass scores CER 0.0890 / WER 0.2661, consistent
with the 1000-sample draw. The validation split is small (~67 pages), so the absolute
numbers are somewhat noisy.
Per class
| class | CER | WER | accuracy | n |
|---|
| handwritten | 0.0750 | 0.2461 | 0.4374 | 958 |
| printed | 0.2996 | 0.6667 | 0.0000 | 11 |
| annotation | 0.4568 | 0.9118 | 0.2917 | 24 |
| table | 0.4917 | 0.9286 | 0.2857 | 7 |
We also acknowledge that printed/table/annotation n is quite small, so measuring CER against them is quite noisy.
Limitations & biases
- Handwritten Ukrainian archival document material only; expect degradation on other
scripts, languages, or modern born-digital text.
- The character set and normalization are tuned to the document-OCR metric — outputs are
normalized text, not faithful transcription.
table is weak and annotation/printed are high-variance (small n).- Single seed and validation split — no across-run variance estimate.
Training data & attribution
