Static GGUF quantizations of StarDoc-AI/NaviDC-OCR for high-speed local document parsing, formula recognition, table transcription, and structured Markdown OCR using llama.cpp.

NaviDC-OCR is a specialized document vision-language model trained to parse complex digital and camera-captured documents, extracting headings, body text, LaTeX math formulas, and Markdown tables with geometric precision.

📊 Benchmark & Evaluation Results

Benchmarked on NVIDIA RTX GPU (CUDA) using llama-server on a complex technical document page containing headings, Schrödinger quantum equations, and multi-column tables:

Quantization	Model Size	BPW	Generation Speed	Prompt Speed	Latency	Accuracy / Elements	Recommended
`NaviDC-OCR-Q4_K_M.gguf`	461.8 MB	4.34	211.4 tok/s	1,320.4 tok/s	1.22s	100% (8/8 elements)	🏆 Best Overall
`NaviDC-OCR-Q5_K_M.gguf`	525.8 MB	5.81	205.7 tok/s	1,262.8 tok/s	1.26s	100% (8/8 elements)	⭐️ High Quality
`NaviDC-OCR-Q8_0.gguf`	767.5 MB	8.50	195.5 tok/s	1,243.4 tok/s	1.30s	100% (8/8 elements)	Reference
`NaviDC-OCR-Q3_K_M.gguf`	394.8 MB	3.50	179.1 tok/s	1,238.5 tok/s	6.26s	Degraded	Lightweight

Companion Multimodal Projector (mmproj):

mmproj-NaviDC-OCR-F16.gguf (1.33 GB) — required for vision encoding alongside any language model quant.

🚀 How to Run with llama.cpp

1. Start `llama-server`

bash

llama-server \
  -m NaviDC-OCR-Q4_K_M.gguf \
  --mmproj mmproj-NaviDC-OCR-F16.gguf \
  --port 8090 \
  -ngl 99 \
  -c 4096

2. Query via OpenAI-compatible API

python

import requests, base64

with open("document_page.png", "rb") as f:
    img_b64 = base64.b64encode(f.read()).decode("utf-8")

response = requests.post(
    "http://127.0.0.1:8090/v1/chat/completions",
    json={
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Extract text."},
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}}
                ]
            }
        ],
        "max_tokens": 2048,
        "temperature": 0.0
    }
)

print(response.json()["choices"][0]["message"]["content"])

📦 Files in this Repository

Filename	Description	Size
`mmproj-NaviDC-OCR-F16.gguf`	Vision Projector (required for all quants)	1.33 GB
`NaviDC-OCR-Q4_K_M.gguf`	Optimal 4-bit Medium Quantization (Fastest & recommended)	461.8 MB
`NaviDC-OCR-Q5_K_M.gguf`	High-fidelity 5-bit Medium Quantization	525.8 MB
`NaviDC-OCR-Q8_0.gguf`	8-bit Reference Quantization	767.5 MB
`NaviDC-OCR-Q3_K_M.gguf`	3-bit Turbo Quantization	394.8 MB
`NaviDC-OCR-F16.gguf`	Unquantized FP16 Baseline	1.50 GB

💡 Acknowledgements & Citation

Original Model: StarDoc-AI/NaviDC-OCR
GGUF Conversion & Benchmarks: Pluto AI by @konradjr007

konradjr007/NaviDC-OCR-GGUF

NaviDC-OCR GGUF Quantizations