This repository contains the DevGen LoRA adapter for Devanagari OCR. It is designed to be loaded on top of paudelanil/trocr-devanagari-2 using the PEFT (Parameter-Efficient Fine-Tuning) library.

Model Details

Developed by: Manish Wagle / DevGen
Base model: paudelanil/trocr-devanagari-2
Adapter type: LoRA
Task: Image-to-text OCR for Devanagari handwritten words and short phrases.
Library: PEFT + Transformers

Intended Use

Use this adapter for high-precision recognition of Devanagari text from cropped handwritten word images. It is particularly optimized for the IIIT-INDIC-HW-WORDS distribution but generalizes well to other handwritten Devanagari styles.

[!IMPORTANT] This model is an OCR engine for word-level recognition. It does not perform page layout analysis, table extraction, or full-page segmentation.

Loading and Inference

To use this model, you need transformers, peft, and torch installed.

python

from peft import PeftModel
from transformers import AutoTokenizer, TrOCRProcessor, ViTImageProcessor, VisionEncoderDecoderModel
import torch
from PIL import Image

base_model_id = "paudelanil/trocr-devanagari-2"
adapter_id = "manishwagle/devgen-trocr-devanagari-lora" # Or local path

# Load processors
image_processor = ViTImageProcessor.from_pretrained(base_model_id)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
processor = TrOCRProcessor(image_processor=image_processor, tokenizer=tokenizer)

# Load model
device = "cuda" if torch.cuda.is_available() else "cpu"
base_model = VisionEncoderDecoderModel.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.to(device)

# Inference
image = Image.open("sample_handwritten_word.png").convert("RGB")
pixel_values = processor(image, return_tensors="pt").pixel_values.to(device)

generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(f"Recognized Text: {generated_text}")

Demo

A hosted Gradio demo and full DevGen OCR Suite interface is available in the Hugging Face Space and the project repository: DevGen Project Repository.

Limitations

Rotation: Sensitivity to severe text rotation (beyond 15 degrees).
Noise: Performance may degrade on heavily blurred or low-contrast scans.
Script: Optimized for Devanagari; may not work for other Indic scripts without additional tuning.

Framework Versions

PEFT 0.19.1
Transformers 4.40.0+
PyTorch 2.1.0+

manishw10/devgen-trocr-devanagari-lora

DevGen TrOCR Devanagari LoRA Adapter