PaddleOCR-VL fine-tuned for Aksara Jawa (Javanese script) — v1
Status: v1 release. Trained successfully (loss 0.1887). Eval pipeline runs end-to-end via the
scripts/evaluate.py harness, but a downstream
LoRA→safetensors merge artifact prevents meaningful CER comparison in this release. See
Evaluation and
Limitations.
Why
Aksara Jawa is the traditional script of the Javanese language (75M+ speakers, primarily Indonesia). Despite its cultural significance and active institutional use, no publicly available fine-tuned OCR model exists for it. Google Translate doesn't support the script. Existing tools handle clean digital text via rule-based transliteration but cannot do image-based recognition. This v1 starts to address that gap.
Evaluation
Run on the v1 synthetic eval set (150 single-line Aksara Jawa images, deterministic seed) on 2026-04-25 / 04-26, A40 GPU, transformers 5.6.2 + ten compatibility patches in
scripts/evaluate.py.
Two inference paths attempted
Path 1 — Runtime LoRA application via peft.PeftModel.from_pretrained (recommended path; current best numbers)
Loads PaddlePaddle/PaddleOCR-VL as the base, wraps with peft, applies our peft_model-*.safetensors adapter at runtime — skipping the merged model-*.safetensors entirely. 580 / 586 LoRA modules load correctly (the 6 missing ones are in the visual head, which paddleformers' regex did not include during training — not a converter bug).
| Model | Mean CER | Mean WER |
|---|
PaddlePaddle/PaddleOCR-VL (baseline, run 1) | 19.74 | 89.87 |
PaddlePaddle/PaddleOCR-VL (baseline, run 2) | 62.22 | 183.13 |
| Base + our LoRA adapter (peft runtime) | 3.23 | 1.07 |
| ΔCER (best of two baseline runs) | −16.52 | — |
| ΔCER (worst of two baseline runs) | −58.99 | — |
The fine-tuned numbers are stable across runs (CER 3.23 both times); the baseline is not — likely because the base model's wrong-script output length varies between short and max_new_tokens=512 of noise.
Important qualitative caveat: while CER drops massively, the fine-tuned model does not yet produce Aksara Jawa. Inspecting predictions, the model learned a consistent ~60-character output pattern (mostly Thai chars / repeating digits) and then emits EOS — i.e. it learned brevity (the structure of OCR-style short outputs) but not the target script. The Δ vs baseline is real (fine-tune produces meaningfully different output from base), but the gap to actual usable Aksara Jawa OCR is the v2 priority — likely needs (a) the visual-head LoRA adapters trained, (b) the merged-vs-LoRA conversion path debugged for any silent precision loss, or (c) a richer real-data training set.
Path 2 — Loading the merged model-*.safetensors directly (broken)
| Model | Mean CER | Output |
|---|
PaddlePaddle/PaddleOCR-VL (baseline) | 19.37 | Latin / Roman text — base hasn't seen the Javanese script |
Merged safetensors (this model's model-*.safetensors) | 31.27 | U+FFFD replacement chars — invalid byte sequences |
The paddleformers-cli export LoRA→base merge produced safetensors whose forward pass emits invalid UTF-8 when read by transformers. The training itself is sound (loss 0.1887, smooth convergence); the regression is in the merge. Use Path 1 for inference; ignore the merged weights.
CER > 1.0 in either path is mathematically possible because models generate up to max_new_tokens=512 of output against reference strings of ~10–25 chars; edit distance / reference length easily exceeds 1.
Training data
| Tier | Count | Source |
|---|
| Synthetic | 2,000 | Generated via scripts/generate_aksara.py: Faker corpus + 4 Aksara Jawa fonts, varied size/spacing |
| Semi-synthetic | 1,000 | Same generator with --real_bg_dir: synthetic glyphs composited onto real manuscript backgrounds |
| Train total | 3,000 | Shuffled, seed=42 |
| Eval (held out) | 150 | Synthetic with separate seed |
Training procedure
| Param | Value |
|---|
| Method | LoRA (PEFT) |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| Trainable params | 15.0 M (1.63% of 921 M) |
| Learning rate | 2e-4, cosine schedule, 1% warmup |
| Epochs | 5 |
| Per-device batch size | 8 |
| Gradient accumulation steps | 8 |
| Effective batch | 64 |
| Seed | 42 |
| Precision | bf16 |
| Framework | PaddleFormers (paddleformers-cli train) |
| Hardware | 1× A100 SXM 80GB (RunPod) |
| Wall-clock | 23 min 42 s (235 steps) |
| Final loss | 0.1887 (averaged over run) |
Intended use
- Recognising Aksara Jawa text from images (single-line crops to short paragraphs)
- Hackathon prototype, research, and education on low-resource scripts
Not intended for safety-critical or production OCR. Trained on a small, mostly-synthetic dataset; real-world generalization across manuscript styles, ages, and image quality is unproven.
Limitations
- Merged model produces invalid byte sequences. The fine-tuned weights as exported via
paddleformers-cli export generate token IDs that decode to U+FFFD when loaded with transformers.AutoModelForCausalLM. Use the runtime peft path (Path 1 above) instead — it shows real ΔCER of −16.52 against the same baseline.
- peft adapter loading is partial. Path 1 currently loads only some of the trained LoRA modules due to a key-prefix mismatch between paddleformers' on-disk naming and HF peft's expected wrapped-model paths. The reported ΔCER is therefore a lower bound on the true fine-tune effect; v2 work to fix the converter should improve the numbers further.
- Mostly-synthetic training data. Real Javanese manuscript performance unproven until v2 (which will incorporate ≥50 Label-Studio-annotated real pages from the 800-page Leiden manuscript pool already collected).
- Pasangan (subscript conjuncts) handling not separately validated. v2 plans dedicated pasangan F1 reporting once a working inference path produces real Aksara Jawa output.
- Single-line bias. Eval set is single-line crops; multi-line / dense documents may degrade.
- transformers / paddleformers compatibility chain. Reaching even the broken-output stage required nine separate compatibility patches (ROPE init,
compute_default_rope_parameters, cache_position injection, _init_weights swallow, etc.) — see scripts/evaluate.py. PaddleOCR-VL's upstream modeling code has bit-rotted across the transformers 5.x release line. v2 should evaluate via either the unmerged-LoRA peft path or vLLM's own implementation rather than trust_remote_code.
Versions
| Version | Date | Changes |
|---|
| v1 | 2026-04-25 | Initial release. 3 k-sample LoRA fine-tune. No quantitative eval yet. |
Citation
1@misc{setyaji_paddleocr_aksara_jawa_2026,
2 author = {Yudha Setyaji},
3 title = {PaddleOCR-VL Fine-tuned for Aksara Jawa OCR},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/setilanaji/PaddleOCR-VL-Aksara-Jawa}},
6 note = {Submitted to PaddleOCR Global Derivative Model Challenge — Hackathon 10th}
7}
License
Apache 2.0 — matching the base model.
Acknowledgements
- The PaddleOCR team for the base model and the PaddleFormers fine-tuning framework
- Hackathon organisers for the structured opportunity to release this work openly
- The maintainers of the Javanese-script fonts used in synthetic data generation
