An 8-bit, group-size-64 MLX quantization of PaddlePaddle/PaddleOCR-VL-1.6, for fast on-device inference on Apple Silicon via mlx-vlm. The architecture is unchanged — this repo only re-quantizes the original weights.

8-bit keeps the decoder stable on harder, lower-resource scripts, where 4-bit quantization is more prone to slipping into repetition / hallucination loops. On easy text (Latin / Cyrillic / CJK) 4-bit and 8-bit are equivalent, so 8-bit is the better trade for broad multilingual OCR at a modest size increase.

Produced with mlx-vlm 0.6.3:

Load with mlx-vlm; see the base model card for the OCR prompt / chat template and supported languages.

Apache-2.0, inherited from the base model. All credit for the model goes to the PaddlePaddle / PaddleOCR team — this repository only provides an MLX-quantized copy of their released weights.