Full-precision
google/gemma-4-26B-A4B-it (mixture-of-experts, 4B active
parameters per token) quantized to 8-bit with imatrix calibration ("oQ8e")
using
oMLX's own quantization tooling,
converted from Google's official weights (not the already-MLX
mlx-community mirror).
Note: with 128 experts and a fixed calibration sample budget, a portion of
rarely-activated experts received few or zero calibration samples during
imatrix collection; those fall back to non-calibrated quantization. This is
an inherent characteristic of MoE + imatrix calibration, not specific to this
model.
Gemma 4's MTP is implemented as a separate small "assistant"/drafter model
(not tensors baked into the main checkpoint), per Google's own design. Pair
this model with
dynamicagency/gemma-4-26b-a4b-it-oQ8e-mtp-drafter
for speculative decoding. In oMLX, set on this model:
1{
2 "vlm_mtp_enabled": true,
3 "vlm_mtp_draft_model": "<local id of the drafter model>"
4}
Run on an Apple M3 Ultra (256GB), full pipeline: guidellm speed, long-context
sweep, lm-evaluation-harness intelligence suite, promptfoo practical suite.
MTP/speculative decoding was active for these runs (drafter attached and
verified via server logs before benchmarking).
6/7 pass (coding 2/2, reasoning 2/2, instruction-following 2/2, summarization 0/1).
