Qwen3.6-27B (dense) quantized to native MXFP4 for Apple Silicon, with the
vision tower and the native Multi-Token-Prediction head preserved and enabled.
This is the smallest bundle in the line — full Qwen3.6-27B capability in 14 GB.
native head preserved, enabled (num_nextn_predict_layers=1)
Quantization
4-bit affine linears via MLX-native mx.quantize (mode="mxfp4",
group_size=32). Norms, hybrid-attention control tensors and the full
vision tower are kept in fp16 passthrough. MTP linears are quantized to
MXFP4; MTP norm/control tensors stay fp16.
Multi-Token Prediction
This bundle keeps Qwen3.6's native MTP module and runs it as a
self-speculative draft head: the MTP head proposes tokens that the main
model verifies in a single pass, so decoded output stays bit-identical to
plain autoregressive decoding — only faster.
Recorded on an M5 Max (vMLX runtime, 96-token deterministic prompt,
output verified equal to baseline at every depth):
Draft depth
tok/s
Speedup
Baseline (MTP off)
24.7
1.00×
D1
40.5
1.64×
D2 (default)
45.7
1.85×
D3
45.0
1.83×
On this bundle D2 is the fastest depth — D3 draws even but does not pull
ahead, so the runtime selects D2 by default.
Absolute tok/s depends on free memory and system load. The speedup
ratio — baseline vs. MTP measured back-to-back under identical
conditions — is the stable figure.
Vision, MTP and caching together
This bundle runs image/video input, native MTP speculative decode and
prefix/KV caching in the same session — a combination not every MTP-enabled
Qwen build exposes. A recorded smoke test confirms both a text prompt and an
image color prompt return correct answers through the combined MTP + VL
runtime.
Loading
Loads via stock MLX tooling on Apple Silicon — the mxfp4 weights are
native mx.quantize affine, no JANG runtime required for the core model.
