MLX (Apple Silicon) MXFP4 quantization of microsoft/Mage-VL, a ~5B vision-language model (Mage-ViT codec-native visual encoder + Qwen3-4B-Instruct text backbone).

This is a full vision-language quant — not text-only. The Qwen3 language backbone is quantized to MXFP4 (OCP microscaling FP4, group size 32); the Mage-ViT vision tower is kept in bf16 (the mlx-vlm default for multimodal towers) to preserve image understanding. This is the smallest / fastest build.

	value
Precision (LLM)	MXFP4, group 32
Vision tower	bf16 (not quantized)
Bits/weight (overall)	5.07 bpw
On-disk size	~2.8 GB
Decode speed¹	62.7 tok/s
Peak memory¹	~4.5 GB

¹ Measured on an Apple M5 Pro (24 GB), single image prompt, 100-token greedy decode. Decode speed is LLM-bound (the vision tower runs once at prefill).

Speed comparison (same machine, full model):

build	decode	peak RAM
bf16	30.1 tok/s	10.9 GB
MXFP8	48.4 tok/s	6.6 GB
MXFP4 (this)	62.7 tok/s	4.5 GB

Verification

Quantization was verified against the original torch model (bf16, CPU). The MLX vision tower (bf16) matches the reference vision features (cosine 0.99992, fp32-vs-fp32). End-to-end, this MXFP4 build stays image-grounded and correct — e.g. it identifies the wizard/book cover art, and on the Mage-VL framework diagram produces:

"A diagram showing the process of predicting gate signals, generating commentary, and decoding text in a machine learning model."

⚠️ 4-bit note: with vision in bf16 the model perceives images accurately, but the 4-bit LLM can be more verbose / run on for short open-ended prompts (e.g. "describe in one sentence"). For strict fidelity to the bf16 model, prefer the MXFP8 sibling.

Usage

⚠️ These are MLX weights (Apple Silicon), for mlx-vlm — not loadable with 🤗 Transformers. mage_vl support is added in mlx-vlm PR #1751. Once it merges, plain pip install -U mlx-vlm is enough. Until then, either install from the PR branch, or copy the bundled model definition into your mlx-vlm (one-time), then use mlx_vlm.generate normally:

bash

# option A — install mlx-vlm with mage_vl support from the PR
pip install "git+https://github.com/SahilChachra/mlx-vlm.git@add-mage-vl"

bash

# option B — stock mlx-vlm + copy the bundled definition (one-time)
pip install mlx-vlm
python - <<'PY'
import mlx_vlm, os, shutil
dst = os.path.join(os.path.dirname(mlx_vlm.__file__), "models", "mage_vl")
shutil.rmtree(dst, ignore_errors=True); shutil.copytree("mlx_vlm_mage_vl", dst)
print("installed mage_vl ->", dst)
PY

# native mlx-vlm CLI (verified)
python -m mlx_vlm generate --model . --image your_image.jpg \
  --prompt "Describe this image." --max-tokens 128 --temperature 0.0

A standalone run_mage_vl_mlx.py (uses the model's HF processor for preprocessing) is also bundled as an alternative.

Notes / limitations

Image path verified. Video inference (the DCVC neural codec) is not included here; copy neural_codec/ from the base repo if you need it.
The streammind_gate (streaming EPFE) component is omitted — not used on the standard image/video path.

Attribution

Base model: microsoft/Mage-VL (Apache-2.0)
Quantized to MLX MXFP4 by @sahilchachra using mlx-lm / mlx-vlm.

sahilchachra/mage-vl-mxfp4-mlx

Mage-VL — MLX MXFP4

Verification

Usage

Notes / limitations

Attribution

sahilchachra/
mage-vl-mxfp4-mlx