MXFP8 provides near-native quality with ~2x compression vs fp16 — better than traditional 8-bit quantization (lower perplexity, higher benchmark scores).

Variants

Variant	Size	Quality	Link
fp16	~4.5 GB	Highest	fp16
MXFP8	~2.6 GB	Near-native	This repo
MXFP4	~1.7 GB	Good	mxfp4

Usage

bash

pip install mlx-vlm

# Text generation
python -m mlx_vlm.generate \
  --model AITRADER/Huihui-Qwen3.5-2B-abliterated-mxfp8-MLX \
  --prompt "Describe this image in detail" \
  --image <path-or-url>

# Chat UI
python -m mlx_vlm.chat_ui \
  --model AITRADER/Huihui-Qwen3.5-2B-abliterated-mxfp8-MLX

Credits

Original model: huihui-ai
Base architecture: Qwen
MLX conversion: AITRADER