MLX 8-bit quantization of PaddlePaddle/PaddleOCR-VL-1.6, a 0.9B-parameter vision-language model for document parsing. Converted for Apple Silicon, running locally with mlx-vlm.

Model details

Base model: PaddlePaddle/PaddleOCR-VL-1.6 (0.9B, ERNIE-4.5-based)
Quantization: 8-bit (MLX mlx_vlm.convert), ~1.09 GB on disk
Architecture: PaddleOCRVLForConditionalGeneration (custom paddleocr_vl model type; trust_remote_code required)
Context: up to 131,072 tokens
Vision encoder: 14px patches, up to ~1M pixels per image

Capabilities

Full-page OCR in Markdown and JSON formats
Layout analysis, table recognition (structure), formula recognition (LaTeX)
Chart and seal recognition, text spotting (detection + recognition)

Intended uses

On-device document parsing on Apple Silicon (M-series)
Local OCR pipelines where no data may leave the machine
Served via mlx_vlm.server or the MLX Python API with trust_remote_code=True

Usage

pip install mlx-vlm

python

from mlx_vlm import load, generate

model, processor = load("OpenGryd/PaddleOCR-VL-1.6-MLX-8bit")
output = generate(model, processor, "media/document.png", max_tokens=4096)
print(output)

To serve OpenAI-compatible endpoints:

python -m mlx_vlm.server --model OpenGryd/PaddleOCR-VL-1.6-MLX-8bit --port 8090

Limitations

8-bit quantization introduces a small accuracy delta vs. the bf16/fp16 base model
Requires trust_remote_code=True (custom modeling code ships with the repo)
No text-only inputs: image required

Evaluation

The base model achieves 96.3% on OmniDocBench v1.6, leading on text, formula, and table recognition. See the base model card for full benchmarks.

License

Apache-2.0 — inherited from the base model.

Citation

bibtex

@misc{paddleocrvl2025,
  title={PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training},
  author={PaddlePaddle Team},
  year={2025},
  howpublished={\url{https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6}},
}