You can deploy this model and start earning money today!

sair390/
tamil-ocr-qwen25vl-lora

Name: tamil-ocr-qwen25vl-lora
Availability: InStock

Views

No views yet

Tamil OCR — Qwen2.5-VL-3B LoRA Adapter

LoRA adapter fine-tuned on Qwen2.5-VL-3B-Instruct for Tamil optical character recognition. Attach to base model for inference. Merged model: sair390/tamil-ocr-qwen25vl

Model Details

Field	Value
Base model	`unsloth/qwen2.5-vl-3b-instruct-unsloth-bnb-4bit`
Architecture	Qwen2.5-VL (3B parameters, vision-language)
Fine-tuning technique	QLoRA (4-bit NF4 quantization)
LoRA rank (r)	64
LoRA alpha	64
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
PEFT version	0.19.0
Language	Tamil (ta)
Task	OCR / image-to-text

Training Hyperparameters

Hyperparameter	Value
Epochs	3
Per-device batch size	1
Gradient accumulation steps	16
Effective batch size	16
Learning rate	2e-4
LR scheduler	Cosine
Warmup ratio	0.03
Weight decay	0.01
Max sequence length	1024
Precision	bf16
Gradient checkpointing	Yes
Save / eval steps	500
Quantization	4-bit NF4, bfloat16 compute
Inference temperature	1.0 (set to 0 for deterministic OCR)

Training Data

Split	Samples	Source
Train	159,939 (80%)	Synthetic + real book scans
Validation	19,992 (10%)	Synthetic + real book scans
Test	19,992 (10%)	Synthetic + real book scans
Total	199,925

Synthetic data augmentations: blur, noise, rotation, brightness/contrast, JPEG artifacts, erosion, shadow overlays Real data: ~150 Tamil books, ~75K pages Fonts: NotoSerifTamil, NotoSansTamil, MuktaMalar, HindMadurai, Catamaran Text sources: Tirukkural, Wikipedia sentences, digitized book corpus Text layouts: 75% single-line, 25% multi-line Font sizes: 24–72pt (single-line), 24–48pt (multi-line)

Usage

python

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
from PIL import Image
import torch

# Load base model (4-bit quantized)
base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "unsloth/qwen2.5-vl-3b-instruct-unsloth-bnb-4bit",
    load_in_4bit=True,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("unsloth/qwen2.5-vl-3b-instruct-unsloth-bnb-4bit")

# Attach LoRA adapter
model = PeftModel.from_pretrained(base_model, "sair390/tamil-ocr-qwen25vl-lora")

# Inference — use temperature=0 for deterministic OCR output
image = Image.open("tamil_page.jpg")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "Read the Tamil text in this image."},
    ],
}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0,        # deterministic — best for OCR
        do_sample=False,
    )

result = processor.decode(output[0], skip_special_tokens=True)
print(result)

4-Stage Inference Pipeline

Preprocessing — deskew + adaptive threshold binarization
Layout analysis — contour detection, bounding box merge, top-to-bottom sort
OCR — Qwen2.5-VL with this LoRA adapter
Post-correction — Tamil vowel sign fixes, Unicode filter, common OCR confusions

Inference Requirements

VRAM: ~6–10 GB (4-bit quantized base)
Dependencies: transformers, peft, torch, Pillow

Limitations

Optimized for printed Tamil text; handwritten text not tested
Performance on degraded/low-resolution scans may vary
Early checkpoint; full training run in progress