Muse-Glimmer-30B — MLX q4-affine
A 4-bit affine MLX quantization of
meta-models/Muse-Glimmer-30B
— the text tower (a Gemma-family multimodal; the text model is served here) — for
MTPLX on Apple Silicon. Pairs with a
dflash
block-diffusion speculative-decoding drafter (see the MTPLX PR linked below).
Architecture
Custom Gemma-3-like text tower (reference: llama.cpp src/models/muse-glimmer.cpp):
- sigmoid gated attention (
self_attn.gate_proj, before o_proj)
- parameter-free QK-norm with
qk_scale_factor folded onto Q
- NoPE on the global/full-attention layers (RoPE θ=500000 only on sliding layers)
- Gemma
(1+w) sandwich norms; RMS-normed embeddings (not ×√hidden); SiLU SwiGLU; logit_scale + tanh softcap
52 layers · hidden 6656 · GQA 32/2 · head-dim 128 · vocab 202048 · sliding-window 2048 (pattern-4) · untied lm_head.
Quantization
q4-affine mirroring unsloth's UD-Q4_K_XL recipe: 4-bit gs32 body; 5-bit lm_head + attention-output
layers 45–51; norms full-precision. 5.06 bpw / 16 GB.
Quality
Base-model completion, greedy, through the real MTPLX /v1/completions serve path (batch-decode,
so indentation is correct):
| benchmark | pass@1 |
|---|
| HumanEval-164 | 47.6% |
| HumanEval+ (stricter) | 43.3% |
| MBPP-378 | 68.8% |
| MBPP+ (stricter) | 56.9% |
Speculative decoding (dflash)
Paired with a bf16 dflash block-diffusion drafter through MTPLX's MTP route. Decode is
token-exact vs greedy AR. Throughput × block size (K), Python code continuation, M5 Max:
| workload | AR tok/s | best K | best tok/s | ×AR |
|---|
| repetitive code (1024 ctx) | 27.2 | 16 | 59.3 | 2.18× |
| short code prompt | 28.1 | 6 | 43.3 | 1.54× |
| diverse code (1024 ctx) | 27.4 | 4 | 28.6 | 1.04× |
Speedup is acceptance-bound: ~2.2× on predictable/boilerplate code, ~break-even on genuinely diverse
code (where the optimum shifts to small K). Full per-K matrix in the MTPLX PR.
Usage (MTPLX)
1mtplx serve --model OpensourceWTF/Muse-Glimmer-30B-MLX-q4 \
2 --stock-ar --no-mtp --unsafe-force-unverified --yes
For speculative decoding, use the dflash pair bundle (target/ + drafter/ + dflash_pair.json)
described in the PR.
Links
- MTPLX integration (arch + dflash backend): https://github.com/youssofal/MTPLX/pull/248
