easy_deepocr/
├── config.json              # Model configuration
├── llm/                     # Qwen2-VL-8B language model weights
├── mm_projector/            # Multimodal projection layer
├── sam_clip_ckpt/           # SAM and CLIP vision encoder weights
└── trainer_state.json       # Training state information

Usage

python

# TODO: Add your inference code here
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("pkulium/easy_deepocr", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("pkulium/easy_deepocr")

# Example inference
# image = ...
# text = ...

Training Details

Base Model: Qwen2-VL-8B
Vision Encoders: SAM + CLIP
Training Framework: VILA
Training Type: Pretraining for OCR tasks

Intended Use

This model is designed for:

Document OCR
Scene text recognition
Handwriting recognition
Multi-language text extraction

Limitations

[Add any known limitations]
Model performance may vary with image quality
Best suited for [specify use cases]

Citation

If you use this model, please cite:

bibtex

@misc{easy_deepocr,
  author = {Ming Liu},
  title = {Easy DeepOCR - VILA-Qwen2-VL-8B},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/pkulium/easy_deepocr}
}

Acknowledgments

VILA for the architecture
Qwen2-VL for the language model
SAM and CLIP for vision encoding capabilities