Unofficial, fully benchmarked GGUF conversion of ATH-MaaS/OvisOCR2, a compact 0.8B page-level document parser. Built and tested with llama.cpp b10502 (0adcc3b).

Q8_0 model + Q8_0 mmproj scored 96.4451 on the full 1,651-page OmniDocBench v1.6 benchmark. This exceeds the 96.34 leading row in the public OmniDocBench table available on 2026-08-23, but is an independent reproduction, not an official leaderboard submission. The upstream model reports 96.58.

Recommended combinations

Use case	Model + mmproj	Total size	Overall
Best quality	`Q8_0` + `mmproj-Q8_0`	884.8 MiB	96.4451
Best balance	`Q4_K_M` + `mmproj-Q8_0`	615.4 MiB	95.9640
BF16 reference	`BF16` + `mmproj-BF16`	1,644.2 MiB	96.4017
Extreme compression	`Q2_K` + `mmproj-Q8_0`	513.4 MiB	87.0212; not recommended

Files

Type	Available files
Language models	`BF16` (1,446.5 MiB), `Q8_0` (774.2), `Q6_K` (600.6), `Q5_K_M` (551.2), `Q4_K_M` (504.8), `Q3_K_M` (444.6), `Q2_K` (402.8 MiB)
IQ2 models	`IQ2_M` (360.4 MiB), `IQ2_XXS` (336.0 MiB)
Vision projectors	`mmproj-OvisOCR2-BF16.gguf` (197.7 MiB), `mmproj-OvisOCR2-Q8_0.gguf` (110.6 MiB)
Reproducibility	`OvisOCR2-imatrix.gguf`, `checksums.sha256`, `CONVERSION_NOTES.md`, `benchmark-results/`

Q6/Q5/Q3/IQ2 were load-tested but not run through the full benchmark.

OmniDocBench results

All GGUF rows used the official end-to-end evaluator on OmniDocBench v1.6_full (1,651 pages). Edit metrics are lower-is-better; others are higher-is-better.

Q8_0 mmproj

Language model	Overall ↑	Text Edit ↓	Formula CDM ↑	Table TEDS ↑	TEDS-S ↑	Read Order Edit ↓	Latency
Q8_0	96.4451	0.02445	97.1602	94.6205	96.9901	0.10951	13.70 s
Q4_K_M	95.9640	0.02499	96.7962	93.5951	95.9249	0.11168	13.34 s
Q2_K	87.0212	0.07611	82.8751	85.7996	88.2746	0.16162	34.75 s

BF16 mmproj

Language model	Overall ↑	Text Edit ↓	Formula CDM ↑	Table TEDS ↑	TEDS-S ↑	Read Order Edit ↓	Latency
BF16	96.4017	0.02520	97.1184	94.6067	96.9301	0.11106	16.90 s
Q8_0	96.2813	0.02503	97.1225	94.2248	96.5677	0.11107	16.23 s
Q4_K_M	95.9722	0.02431	96.7846	93.5631	95.9050	0.11077	14.86 s
Q2_K	86.8312	0.07677	82.4504	85.7206	88.2972	0.16092	35.05 s

For reference, our upstream HF BF16/vLLM run scored 96.5943. Q2_K generated valid output for 1,649/1,651 pages and hit the 16,384-token limit on 203 pages; the two failed pages were scored as empty predictions. Full CSV/JSON evidence is in benchmark-results/.

Official leaderboard context

Selected leading rows from the OmniDocBench v1.6_full public table:

Method	Size	Overall ↑
PaddleOCR-VL-1.6	0.9B	96.34
MinerU2.5-Pro	1.2B	95.75
GLM-OCR	0.9B	95.22

Evaluation setup

Hardware: RTX 5060 Ti 16 GB; Ryzen 7 7700; 32 GB RAM.
System: Windows 11 + WSL2 Ubuntu 24.04; CUDA 13.3; Python 3.12.3.
Runtime: llama.cpp b10502, commit 0adcc3b; all layers on GPU.
Generation: official OvisOCR2 prompt; temperature 0; thinking off; maximum 16,384 output tokens; 448–2880 px image range (196–8100 vision tokens).
Server: 3 continuous slots; 32K context per slot; batch 8192; ubatch 2048; Flash Attention; BF16 KV cache; speculative/MTP decoding off.
Evaluator: official end-to-end quick-match evaluator, 4 workers. GPU metrics were sampled every second.

Run with llama.cpp

Use llama.cpp b10502 or newer:

bash

./llama-server \
  -m OvisOCR2-Q4_K_M.gguf \
  --mmproj mmproj-OvisOCR2-Q8_0.gguf \
  -ngl all --ctx-size 32768 --flash-attn on \
  --image-min-tokens 196 --image-max-tokens 8100 --jinja

The source model declares an unpublished MTP/NextN layer, so conversion uses --no-nextn; no published OCR weights are removed. See CONVERSION_NOTES.md for commands and compatibility details.

Links and license

Apache-2.0. This is an unofficial conversion; verify critical OCR output manually.

MDfox/OvisOCR2-GGUF-SOTA-20260823

OvisOCR2 GGUF SOTA — 2026-08-23