You can deploy this model and start earning money today!

zenosai/
MonkeyOCRv2-AS

Name: MonkeyOCRv2-AS
Availability: InStock

transformers

safetensors

monkeyocrv2_vitae_encoder

feature-extraction

image-feature-extraction

Views

No views yet

MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

Project Page

arXiv

MonkeyOCRv2

MonkeyDocv2

Demo

News

2026.09.07 🚀 MonkeyOCRv2-Parsing is now supported by RAGFlow as a PDF parser. Thanks to the RAGFlow team and community for the support!
2026.08.22 🖥️ We now support running MonkeyOCRv2-Parsing on CPU. See the CPU support instructions.
2026.08.17 ⚡ Releasing training and evaluation instructions for Recognition, Detection, Overlapping Text Segmentation and Formula.
2026.07.24 ⚡ We released MonkeyOCRv2-B-Parsing-DFlash, enabling vLLM serving with DFlash for up to 2× faster inference.
2026.07.22 🏆 MonkeyOCRv2-B-Parsing ranks #1 among evaluated open-source models on the official MDPBench Leaderboard, achieving 83.3 overall across 17 languages, including digital-born and photographed documents.
2026.07.21 📦 We release MonkeyDoc v2, an open multilingual corpus for document-oriented pretraining. We hope it can serve as a shared data foundation for more transparent, reproducible, and fair comparisons in Document AI.
2026.07.14 🚀 We release MonkeyOCRv2, including MonkeyOCRv2 vision encoder, MonkeyOCRv2-Parsing for multilingual document parsing, MonkeyOCRv2-Und for efficient document understanding.

Use MonkeyOCRv2 as a Vision Backbone

MonkeyOCRv2 is released as a standalone, document-native vision encoder. It can be integrated into different OCR and document AI systems as a visual backbone.

The current release has been evaluated on document parsing, document understanding, text recognition, formula recognition, text detection, document tampering detection, and overlapping-text segmentation.

Beyond these evaluated tasks, the encoder may also be useful for text-rich scenarios such as scientific papers, historical documents, medical reports, charts and tables, and remote-sensing maps or reports. We welcome community exploration of these directions.

python

from transformers import AutoModel

encoder = AutoModel.from_pretrained(
    "zenosai/MonkeyOCRv2-B",
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
)

See the Vision Encoder Quick Start for installation and feature-extraction examples. If you adapt MonkeyOCRv2 to a new task or domain, feel free to open an issue or pull request and share the results.

MonkeyDoc v2

MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released 52 million synthetic samples and 52 million real-world samples. You can download the full dataset as follows:

bash

pip install modelscope
modelscope download --dataset zenosai/MonkeyDocv2 --local_dir ./MonkeyDocv2

After processing and compression, downloading the dataset currently requires approximately 10 TB of disk space. We recommend having at least 11 TB of available storage before starting the download to ensure sufficient space throughout the process.

Model Zoo

1. Vision Encoder

Model	Backbone	Params	Pretraining Resolution	Applicable Tasks	Checkpoint Link
Monkey OCRv2-S	ViT-S	28M	12802828	Recognition / Parsing / Understanding	🤗HuggingFace 🤖ModelScope
Monkey OCRv2-B	ViT-B	113M	12802828	Recognition / Parsing / Understanding	🤗HuggingFace 🤖ModelScope
Monkey OCRv2-AS	ViTAEv2-S	21M	17603232	Detection / Segmentation	🤗HuggingFace 🤖ModelScope

2. Document Parsing Model

Model	Link	Total Params	ViT	LLM	All	Digit.	Photo.	Latin Avg.	DE	EN	ES	FR	ID	IT	NL	PT	VI	Non-Latin Avg.	AR	HI	JP	KO	RU	TH	ZH	ZH-T
MonkeyOCRv2-S-Parsing	HuggingFace ModelScope	0.6B	0.03B	0.6B	82.5	87.9	80.7	83.2	87.3	83.6	76.8	73.6	85.4	87.2	85.5	87.4	81.9	81.7	91.2	87.1	69.9	88.7	78.0	79.8	84.4	74.7
MonkeyOCRv2-B-Parsing	HuggingFace ModelScope	0.7B	0.1B	0.6B	83.3	88.1	81.7	84.2	87.7	84.5	75.2	78.4	86.5	88.6	86.1	87.9	83.2	82.1	90.7	87.2	71.9	87.6	80.1	80.8	83.6	75.3

3. Document Understanding Model

Model	Link	Total Params	Overall	DocVQA	InfoVQA	DF	KLC	WTQ	ChartQA	DT-VQA	OCRBench
MonkeyOCRv2-S-Und	HuggingFace ModelScope	1.7B	55.9	79.3	44.5	65.1	37.6	43.0	62.0	63.1	52.2
MonkeyOCRv2-B-Und	HuggingFace ModelScope	1.8B	57.2	79.3	46.3	65.8	38.2	43.2	62.0	64.3	58.1

Quick Start

What trust_remote_code=True loads is listed in docs/trust_remote_code.md.

Vision Encoder

1. Install

Install transformers and flash attention:

bash

conda create -n MonkeyOCRv2 python=3.11
conda activate MonkeyOCRv2
pip install torch==2.8.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126
pip install transformers==4.57.1
pip install accelerate==1.11.0
pip install qwen_vl_utils==0.0.14
pip install flash-attn==2.8.3 --no-build-isolation

2. Download Model Weights

Download our model from Huggingface.

python download_model.py -n MonkeyOCRv2-B # or MonkeyOCRv2-S / MonkeyOCRv2-AS

You can also download our model from ModelScope.

bash

pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B # or MonkeyOCRv2-S / MonkeyOCRv2-AS

3. Extract Image Feature

bash

cd vision
# For MonkeyOCRv2-B and MonkeyOCRv2-S
python extract_feature.py -m ../model_weight/MonkeyOCRv2-B -i ../images_test/ar.JPEG
# For MonkeyOCRv2-AS
python extract_feature_vitae.py -m ../model_weight/MonkeyOCRv2-AS -i ../images_test/ar.JPEG

Document Parsing

1. Install

Install vLLM following its official guide:

bash

conda create -n MonkeyOCRv2Parsing python=3.11
conda activate MonkeyOCRv2Parsing
pip install uv
uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.25.1/cu129 --extra-index-url https://download.pytorch.org/whl/cu129 -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install -r parsing/requirements.txt

To use DFlash for faster inference, vLLM 0.25.1 is required, which depends on CUDA 12.9 or later.

If your system does not support CUDA 12.9, you can instead install vLLM 0.11.2 (without DFlash support) by running:

uv pip install vllm==0.11.2 --torch-backend=auto -i https://pypi.tuna.tsinghua.edu.cn/simple

Inference will still work normally, but DFlash acceleration will not be available.

2. Download Model Weights

Download our model from Huggingface.

bash

python download_model.py -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
# use DFlash for faster inference, support MonkeyOCRv2-B-Parsing only for now
python download_model.py -n MonkeyOCRv2-B-Parsing-DFlash

You can also download our model from ModelScope.

bash

pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
# use DFlash for faster inference, support MonkeyOCRv2-B-Parsing only for now
python download_model.py -t modelscope -n MonkeyOCRv2-B-Parsing-DFlash

3. vLLM Serving

You should start a vLLM service before parsing documents:

bash

cd parsing
# Serve with DFlash for faster inference
python serve.py -m ../model_weight/MonkeyOCRv2-B-Parsing -d ../model_weight/MonkeyOCRv2-B-Parsing-DFlash -p 8888
# Serve without DFlash
python serve.py -m ../model_weight/MonkeyOCRv2-B-Parsing  -p 8888
# 2 GPUs
python serve.py -m ../model_weight/MonkeyOCRv2-B-Parsing --tp 2 -p 8888
# Show help messages
python serve.py -h

4. Inference

You can parse documents using CLI or serve with demo and FastAPI.

4.1 Parse using CLI

Parse a single document or a directory containing PDFs or images:

bash

cd parsing
python parse.py \
    -i ../images_test \
    -o output/test \
    -s http://127.0.0.1:8888 \
    --draw-layout \
    --skip-processed
# Show help messages
python parse.py -h

4.2 Serve with Web Demo

bash

cd parsing
python demo/gradio_demo.py -s http://127.0.0.1:8888 -p 8891
# Show help messages
python demo/gradio_demo.py -h

You can access the web demo at http://localhost:8891.

4.3 Serve with FastAPI

bash

cd parsing
python fastapi/main.py -s http://127.0.0.1:8888 -p 8000
# Show help messages
python fastapi/main.py -h

You can access the API documentation at http://localhost:8000/docs to explore available endpoints.

POST /parse accepts a single multipart field file or a repeatable files field and returns application/zip (monkeyocrv2_results.zip), with one directory per document containing Markdown. See /docs for the remaining endpoints and request fields.

5. Fine-tune

You can fine-tune MonkeyOCRv2-Parsing on your own data. Please refer to the training instructions.

Document Understanding

1. Install

You can use vLLM for faster inference. Install vLLM following its official guide:

bash

conda create -n MonkeyOCRv2Und python=3.11
conda activate MonkeyOCRv2Und
pip install uv
uv pip install vllm==0.11.2 --torch-backend=auto -i https://pypi.tuna.tsinghua.edu.cn/simple

or follow the install part of MonkeyOCRv2 vision encoder if you use transformers for aligned performance with the paper.

2. Download Model Weights

Download our model from Huggingface.

python download_model.py -n MonkeyOCRv2-B-Und # or MonkeyOCRv2-S-Und

You can also download our model from ModelScope.

bash

pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Und # or MonkeyOCRv2-S-Und

3. Inference

3.1 Infer with vLLM

Start vLLM service first:

bash

cd understanding
python serve.py -m ../model_weight/MonkeyOCRv2-B-Und  -p 8889

Run inference scripts:

bash

python infer.py \
    -s http://127.0.0.1:8889 \
    -i ../images_test/vqa.png \
    -q 'What is the serving size?'
# Show help messages
python infer.py -h

3.2 Infer with transformers

You can infer with transformers for aligned performance with the paper:

bash

python infer.py \
    -m ../model_weight/MonkeyOCRv2-B-Und \
    --attn-implementation flash_attention_2 \
    -i ../images_test/vqa.png \
    -q 'What is the serving size?'

4. Fine-tune

You can fine-tune MonkeyOCRv2-Und on your own data. Please refer to the training instructions.

Scene Text Recognition

Please refer to ./recognition for training and evaluation instructions.

Scene Text Detection

Please refer to ./detection for training and evaluation instructions. Detection is split into two trees: DPText-DETR/ and mmocr/ (DBNet and PSENet).

Overlapping Text Segmentation

Please refer to MOT for training and evaluation instructions.

Formula Recognition

Please refer to ./formula for training and evaluation instructions.

Document Tampering Detection

Please refer to ./forensics/readme.md for installation, checkpoints, and evaluation.

Visualization

Our model supports robust document parsing in real-world scenarios across 17 languages, including Simplified Chinese (ZH), Traditional Chinese (ZH-T), English (EN), Arabic (AR), German (DE), Spanish (ES), French (FR), Hindi (HI), Indonesian (ID), Italian (IT), Japanese (JP), Korean (KO), Dutch (NL), Portuguese (PT), Russian (RU), Thai (TH), and Vietnamese (VI).

Evaluation Results

1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.

Model	Overall	Union14M-Benchmark	Chinese Benchmarks	Occlusion SceneText
Avg	Artistic	Context less	Curve	General	Multi Oriented	Multi Words	Saliency	Avg	Scene	Web	Document	Hand writing
ABINet	73.7	75.7	71.7	74.7	80.4	79.8	69.0	76.8	77.6	70.3	66.6	63.2	98.2	53.1	75.0
MAERec	81.6	85.2	79.0	84.2	89.1	84.6	87.1	85.9	86.3	83.1	84.4	83.0	99.5	65.6	76.4
CPPD	80.4	81.9	76.5	82.9	86.2	83.5	78.7	81.9	83.5	81.7	82.7	82.4	99.4	62.3	79.6
IGTR-AR	81.0	84.9	77.0	82.4	90.4	84.4	91.2	84.0	84.7	81.7	82.0	81.7	99.5	63.8	76.3
SMTR	80.4	85.0	76.8	83.9	89.1	83.7	87.7	89.3	84.6	82.7	83.4	83.0	99.3	65.1	73.5
SVTRv2	83.1	86.1	79.3	86.1	90.6	85.1	89.0	86.7	86.2	83.3	83.5	83.3	99.5	67.0	80.0

CRNN (ResNet)	58.7	49.2	51.2	62.3	48.1	68.2	13.0	60.4	41.4	68.8	63.8	68.2	97.0	46.1	58.0
CRNN (MonkeyOCRv2-S)	67.3	65.2	63.7	73.0	71.1	74.5	28.6	72.1	73.4	74.2	73.0	74.9	96.9	51.8	62.4
PARSeq (ViT)	82.2	84.3	76.5	83.4	87.6	84.9	88.8	84.3	84.4	82.4	84.2	82.8	99.5	63.0	79.9
PARSeq (MonkeyOCRv2-S)	84.3	87.6	78.6	86.4	92.1	85.4	93.9	88.7	87.7	83.7	84.6	83.2	99.5	67.3	81.5

2. Formula recognition results on OmniDocBench 1.6, MathWriting, and UniMER-Test.

Model	Params	Overall	OmniDocBench 1.6	MathWriting	SPE	CPE	HWE	SCE
CDM	ExpRate	CDM	ExpRate	CDM	ExpRate	CDM	ExpRate	CDM	ExpRate	CDM	ExpRate	CDM	ExpRate
Pix2tex	25.5M	53.8	23.3	69.4	27.0	0.4	0.0	96.2	72.4	64.9	7.1	24.5	0.6	67.6	32.8
Texify	312M	67.3	40.4	76.5	46.4	26.6	2.0	98.5	91.0	70.4	28.2	52.7	23.6	79.3	51.3
UniMERNet-B	325M	89.5	64.5	90.4	59.5	63.8	12.3	99.1	93.3	96.0	80.5	94.0	64.3	93.7	77.0
UniMERNet-S	202M	89.8	64.0	90.1	59.1	65.9	12.7	99.1	93.4	95.9	77.7	93.7	63.9	94.1	76.9

UniMERNet-T (Swin)	107M	89.4	61.8	89.9	57.2	65.6	12.9	99.1	92.3	94.9	69.9	93.3	61.9	93.8	76.6
UniMERNet-T (MonkeyOCRv2-S)	110M	90.9	66.4	90.8	61.1	70.8	16.2	99.2	93.8	96.1	79.2	94.3	69.5	94.0	78.6

3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of MMOCR and DPText-DETR.

4. Document tampering detection results on DocTamper benchmark.

Method	Params	Overall	DocTamper-Test	DocTamper-FCD	DocTamper-SCD
IoU	F	IoU	P	R	F	IoU	P	R	F	IoU	P	R	F
PSCC-Net	5M	13.7	31.3	17.0	25.0	83.0	39.0	13.0	19.0	82.0	30.0	11.0	15.0	83.0	25.0
UperNet	67M	49.3	54.0	70.0	66.0	60.0	62.0	30.0	57.0	35.0	43.0	48.0	57.0	58.0	57.0
CAT-Net	114M	67.3	71.0	78.0	75.0	69.0	72.0	66.0	85.0	70.0	76.0	58.0	65.0	65.0	65.0
Swin-UPer	81M	66.7	71.7	79.0	75.0	72.0	73.0	64.0	80.0	70.0	75.0	57.0	66.0	68.0	67.0
SegFormer	85M	70.3	74.0	81.0	77.0	74.0	75.0	69.0	82.0	74.0	78.0	61.0	68.0	70.0	69.0
Mask2Former	69M	69.7	78.0	84.0	82.0	83.0	82.0	66.0	81.0	75.0	78.0	59.0	70.0	79.0	74.0
ConvNext	122M	69.7	75.3	84.0	81.0	78.0	79.0	62.0	76.0	71.0	74.0	63.0	71.0	74.0	73.0
ConvNextV2	121M	72.7	77.7	86.0	82.0	79.0	81.0	65.0	79.0	75.0	77.0	67.0	74.0	76.0	75.0
InternImage	128M	73.3	77.7	84.0	81.0	77.0	79.0	72.0	83.0	79.0	81.0	64.0	73.0	74.0	73.0
ASC-Former	80M	68.2	80.8	81.5	91.8	87.8	89.8	61.3	74.9	77.1	76.0	61.9	78.0	75.0	76.5
DTD	66M	77.0	79.7	84.0	81.0	77.0	79.0	79.0	88.0	82.0	85.0	68.0	75.0	76.0	75.0

FFDN* (ViTAEv2)	69M	70.7	82.7	69.4	76.2	88.7	82.0	79.0	92.5	84.4	88.3	63.6	79.1	76.5	77.8
FFDN (MonkeyOCRv2-AS)	71M	78.2	87.5	87.4	94.8	91.8	93.3	79.9	90.4	87.4	88.9	67.2	81.0	79.8	80.4

* denotes models trained with the ViTAEv2 pretrained by DeepSolo

5. Overlapping text segmentation results on MOT dataset.

Model	mIoU_Text	IoU_Occ	IoU_Occd	IoU_Ov
Unet	62.2	80.2	65.7	40.7
Deeplab v3	67.9	83.2	71.2	49.3
OCRNet	65.8	81.0	68.5	47.8
Segformer	69.0	83.6	74.1	49.3
MaskFormer	68.4	83.5	70.3	51.4
TexRNet	68.9	84.2	73.2	49.3
EAFormer	69.1	83.8	74.2	50.5
WASNet	70.8	84.8	74.4	53.1

Mask2Former (ResNet)	70.3	84.7	73.3	52.8
Mask2Former (MonkeyOCRv2-AS)	76.6	88.6	83.4	57.7
MOTS (ResNet)	72.6	85.2	77.5	54.9
MOTS (MonkeyOCRv2-AS)	76.9	88.6	82.6	59.4

6. Document parsing results on MDPBench, a comprehensive multilingual benchmark for real-world document parsing.

Model	Total Params	ViT	LLM	All	Digit.	Photo.	Latin Avg.	DE	EN	ES	FR	ID	IT	NL	PT	VI	Non-Latin Avg.	AR	HI	JP	KO	RU	TH	ZH	ZH-T
Closed-source VLMs
ChatGPT-5.2-2025-12-11	-	-	-	68.6	85.6	63.0	75.2	70.8	79.4	71.4	60.0	77.7	78.5	71.6	85.0	82.1	61.1	64.9	63.4	55.8	65.4	60.7	63.8	56.3	58.7
Claude-Sonnet-4.6	-	-	-	73.1	85.0	69.3	79.2	79.8	80.6	72.8	66.5	82.3	83.3	76.7	88.0	83.1	66.2	67.8	71.7	63.4	64.3	70.8	65.2	61.3	65.1
Doubao-2.0-pro	-	-	-	74.2	78.9	72.8	75.7	82.8	74.4	69.0	70.0	73.3	82.0	69.9	83.4	76.5	72.5	81.3	75.7	65.8	74.7	63.3	71.9	71.9	75.2
Gemini-3-pro	-	-	-	86.4	90.4	85.1	88.4	91.2	90.6	83.4	82.7	91.5	91.6	87.7	91.4	85.9	84.1	89.4	90.4	74.8	85.5	84.9	80.6	85.1	82.1
Open-source VLMs
InternVL-3.5-8B	8.3B	0.3B	8B	42.7	59.7	37.0	53.4	39.8	64.2	47.5	42.7	53.8	60.6	52.2	63.2	57.0	30.6	8.2	9.0	45.6	30.3	26.1	10.8	55.3	59.3
MinerU-2.5	1.2B	0.7B	0.5B	46.3	61.9	40.8	63.0	68.8	78.4	54.7	57.3	67.5	75.2	60.4	58.8	46.0	27.4	1.3	9.0	39.1	14.7	8.6	11.3	72.9	62.2
DeepSeek-OCR	3.4B	0.4B	3B	51.8	80.7	42.2	54.5	55.0	58.3	44.1	43.2	60.9	69.3	52.4	53.0	54.1	48.9	56.9	52.2	49.1	28.2	36.2	49.4	59.7	59.2
MonkeyOCR-pro-3B	3.7B	0.7B	3B	52.2	68.0	47.0	65.1	71.7	77.9	55.9	62.1	66.2	74.5	66.3	71.1	40.2	37.6	4.6	4.2	55.2	60.5	42.6	9.1	72.2	52.4
Nanonets-OCR-s	4.7B	0.7B	4B	63.7	78.8	58.7	71.3	75.1	78.5	61.2	62.5	70.3	81.0	69.6	75.9	67.5	55.0	59.5	61.8	55.9	51.2	43.5	39.5	67.4	61.5
Nanonets-OCR2-3B	3.7B	0.7B	3B	64.2	79.2	59.3	71.4	76.7	76.4	61.8	66.1	68.4	78.5	74.1	74.2	66.0	56.2	60.2	59.2	52.1	54.7	45.5	44.6	68.3	65.1
Qwen3.5-Instruct-9B	9.7B	0.7B	9B	65.7	74.8	62.7	72.5	72.8	72.0	72.0	64.4	66.2	77.6	74.5	79.1	74.0	58.2	53.4	56.2	55.7	60.3	54.7	56.7	60.8	67.5
GLM-OCR	0.9B	0.4B	0.5B	67.3	77.9	63.7	78.7	82.7	84.5	75.8	76.2	79.7	82.8	80.2	77.4	69.2	54.3	21.7	39.6	65.5	61.2	64.2	27.4	78.5	76.7
Qwen3-VL-Instruct-8B	8.3B	0.3B	8B	68.3	78.4	65.0	73.6	73.7	71.4	69.3	66.2	68.5	79.1	78.3	82.2	73.4	62.5	63.1	58.4	59.9	61.9	57.9	62.0	62.6	73.8
HunyuanOCR	1B	0.4B	0.6B	68.3	80.2	64.3	72.4	75.0	73.1	63.0	66.1	69.9	80.3	61.4	81.9	80.6	63.7	68.3	73.1	55.6	68.9	52.2	60.7	66.8	64.2
PaddleOCR-VL	0.9B	0.6B	0.3B	69.6	87.6	63.6	72.1	78.2	79.3	62.9	66.0	77.4	78.4	67.9	72.0	66.6	66.7	65.8	68.4	59.9	77.8	56.9	57.8	78.2	68.5
olmOCR2	7.7B	0.7B	7B	70.4	79.9	67.2	76.7	75.7	77.3	72.5	68.9	70.6	81.0	72.0	88.0	84.0	63.3	59.0	60.8	59.4	70.6	65.8	59.2	68.6	63.4
MinerU-2.5-Pro	1.2B	0.7B	0.5B	71.0	86.2	66.1	74.6	78.3	79.5	63.4	67.4	78.0	79.7	72.1	78.6	74.2	67.0	56.6	72.2	59.1	77.6	62.6	61.8	76.5	69.7
PaddleOCR-VL-1.6	0.9B	0.6B	0.3B	75.0	82.8	72.6	78.0	84.1	79.7	69.2	74.8	81.6	82.0	74.7	76.4	79.3	71.6	69.4	65.6	68.7	82.5	70.7	62.3	78.0	75.7
HunyuanOCR-1.5	1B	0.4B	0.6B	76.8	86.2	73.6	79.7	79.6	80.4	74.2	70.0	81.5	84.5	78.4	86.4	82.4	73.5	71.8	71.6	65.5	75.7	67.4	77.7	80.8	77.2
Kimi-K2.5	1T	0.4B	1T	77.5	85.0	75.0	81.6	85.9	86.2	72.7	71.0	80.6	86.6	77.4	87.6	86.2	72.9	75.8	74.5	72.5	70.9	61.8	67.0	81.7	78.6
PaddleOCR-VL-1.5	0.9B	0.6B	0.3B	78.3	87.4	75.2	81.2	84.8	83.0	75.7	78.1	83.9	85.2	80.6	80.2	78.9	74.9	71.3	67.7	69.5	86.0	76.0	68.4	84.8	75.7
chandra-ocr-2	5.3B	0.5B	4.8B	79.7	87.8	77.1	82.7	86.6	86.5	69.7	70.3	84.6	87.4	82.7	90.7	85.6	76.4	78.2	81.1	68.8	80.3	74.0	78.5	73.8	76.3
dots.mocr	3B	1.2B	1.8B	80.5	90.5	77.2	81.7	82.6	87.4	71.3	70.1	84.5	89.3	83.2	86.8	79.9	79.2	83.3	83.6	75.0	78.7	71.2	77.9	84.6	79.6
MonkeyOCRv2-S-Parsing🤗	0.6B	0.03B	0.6B	82.5	87.9	80.7	83.2	87.3	83.6	76.8	73.6	85.4	87.2	85.5	87.4	81.9	81.7	91.2	87.1	69.9	88.7	78.0	79.8	84.4	74.7
MonkeyOCRv2-B-Parsing🤗	0.7B	0.1B	0.6B	83.3	88.1	81.7	84.2	87.7	84.5	75.2	78.4	86.5	88.6	86.1	87.9	83.2	82.1	90.7	87.2	71.9	87.6	80.1	80.8	83.6	75.3

7. Document understanding performance comparison across different vision foundation models. The evaluation benchmarks are selected following TextMonkey and DT-VQA.

Model	Params	Overall	DocVQA	InfoVQA	DF	KLC	WTQ	ChartQA	DT-VQA	OCRBench
CLIP-B	86M	16.0	20.1	24.2	2.3	13.8	12.8	22.2	22.3	10.6
SigLIP2-B	93M	24.9	27.0	23.5	3.1	16.7	17.4	35.0	41.5	35.1
RADIOv2.5-B	98M	37.5	60.3	31.2	29.9	30.4	29.7	51.1	44.2	23.1
OpenVision-B	87M	44.0	63.3	30.7	19.8	33.1	31.1	58.3	62.6	52.9
DINOv3-B	86M	16.1	26.5	20.8	5.6	13.2	14.0	28.9	15.8	3.9
SAM-B	90M	25.2	37.8	22.2	4.7	17.5	17.6	46.5	33.3	21.9
SAM2-B	69M	22.3	32.5	21.9	2.7	15.8	16.6	40.2	30.3	18.4
oCLIP	24M	12.4	14.8	19.5	1.4	7.4	11.4	17.9	19.2	7.4
DiT	86M	8.9	11.3	20.9	0.9	5.2	9.9	12.0	9.2	1.9
*MonkeyOCRv2-S🤗Link**	28M	55.9	79.3	44.5	65.1	37.6	43.0	62.0	63.1	52.2
*MonkeyOCRv2-B🤗Link**	113M	57.2	79.3	46.3	65.8	38.2	43.2	62.0	64.3	58.1

MonkeyDoc v2

MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released all 52 million synthetic samples and 41 million real-world samples derived from FinePDF. Additional real-world data from other sources is currently being organized and will be released progressively.

Expert Model Labeling Toolchain

We adopt a multi-expert labeling pipeline to obtain reliable annotations for documents. The pipeline includes the following steps:

Structure Detection
We use dots.mocr for document structure detection and reading-order prediction. The detected regions, including text blocks, tables, formulas, and other layout elements, are cropped from the original page image for subsequent recognition.
Content Recognition
Each cropped block is independently recognized by three expert models: dots.mocr, PaddleOCR-VL, and Qwen3-VL. These complementary models provide multiple annotations for the same block, reducing reliance on any single OCR system.
Block-Level Agreement Filtering
We compare the recognition results from the three expert models and filter out blocks with low agreement. For retained blocks, we select the prediction that has the highest average agreement with the other two predictions as the final block-level annotation.
Page-Level Quality Control
Pages containing any filtered block are discarded. In addition, we use Qwen3 to verify whether the predicted reading order is reasonable, and Qwen3-VL to check whether document regions are missed during structure detection. This multi-expert agreement strategy reduces model-specific annotation errors and improves the reliability of the generated annotations.

References

dots.mocr: https://github.com/rednote-hilab/dots.mocr
PaddleOCR-VL: https://github.com/PaddlePaddle/PaddleOCR
Qwen3-VL: https://github.com/QwenLM/Qwen3-VL
Qwen3: https://github.com/QwenLM/Qwen3
MonkeyOCR: https://github.com/Yuliang-Liu/MonkeyOCR
MDPBench: https://github.com/Yuliang-Liu/MultimodalOCR
MonkeyDoc: https://modelscope.cn/datasets/zenosai/MonkeyDoc
MonkeyDoc v2: https://modelscope.cn/datasets/zenosai/MonkeyDocv2

Citation

If you use any part of this release — the MonkeyOCRv2 encoders, MonkeyOCRv2-Parsing, MonkeyOCRv2-Und, the MDPBench benchmark, or the MonkeyDoc v2 dataset — please cite:

bibtex

@article{liu2026monkeyocrv2,
  title   = {MonkeyOCRv2: A Visual-Text Foundation Model for Document AI},
  author  = {Liu, Yuliang and Li, Zhang and Zhang, Ziyang and Zhang, Shuo and
             Liu, Qiang and Song, Jiajun and Guo, Zidun and Wang, Xinhan and
             Zheng, Handong and Liu, Yang and Luo, Dongliang and Ma, Zhiyin and
             Zhang, Jiarui and Bai, Xiang},
  journal = {arXiv preprint arXiv:2607.11562},
  year    = {2026}
}

Copyright

Share reproductions, integrations, and failure cases in GitHub Discussions. We warmly welcome your feedback, suggestions, and contributions, which are essential to the continued development and improvement of our framework. The models are are released under the Apache License 2.0 and are free for both research and commercial use. For any questions, please contact us at xbai@hust.edu.cn or ylliu@hust.edu.cn.

zenosai/MonkeyOCRv2-AS

MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

News

Use MonkeyOCRv2 as a Vision Backbone

MonkeyDoc v2

Model Zoo

1. Vision Encoder

2. Document Parsing Model

3. Document Understanding Model

Quick Start

Vision Encoder

1. Install

2. Download Model Weights

3. Extract Image Feature

Document Parsing

1. Install

2. Download Model Weights

3. vLLM Serving

4. Inference

4.1 Parse using CLI

4.2 Serve with Web Demo

4.3 Serve with FastAPI

5. Fine-tune

Document Understanding

1. Install

2. Download Model Weights

3. Inference

3.1 Infer with vLLM

3.2 Infer with transformers

4. Fine-tune

Scene Text Recognition

Scene Text Detection

Overlapping Text Segmentation

Formula Recognition

Document Tampering Detection

Visualization

Evaluation Results

1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.

2. Formula recognition results on OmniDocBench 1.6, MathWriting, and UniMER-Test.

3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of MMOCR and DPText-DETR.

4. Document tampering detection results on DocTamper benchmark.

5. Overlapping text segmentation results on MOT dataset.

6. Document parsing results on MDPBench, a comprehensive multilingual benchmark for real-world document parsing.

7. Document understanding performance comparison across different vision foundation models. The evaluation benchmarks are selected following TextMonkey and DT-VQA.

MonkeyDoc v2

Expert Model Labeling Toolchain

References

Citation

Copyright

zenosai/
MonkeyOCRv2-AS