Apple Silicon (MLX) mixed-precision quantization of zai-org/GLM-5.2 — a 744B-parameter (~40B active) Mixture-of-Experts model with DeepSeek-V3.2-style MLA + DeepSeek Sparse Attention (DSA, glm_moe_dsa) — pushed to the lowest effective bit-rate the MLX affine container supports (routed experts at 2-bit/g128): 2.3225 bpw measured, 219.75 GB on disk. It is the smallest published GLM-5.2 build in any format we know of, and the first in this family produced with anchor-guarded clip-search + layerwise DWQ stacked.
⚠️ Experimental. This is a floor experiment that ended up genuinely usable. On the same ≤256 GB machine class the 2.56 bpw sibling is still somewhat better (−5.2% wikitext PPL on its own clip+DWQ rework) at +23 GB; pick this build when the extra ~23 GB of headroom matters (longer prefills / KV on a 256 GiB box). Full methodology, the teacher A/B, and the clip-search saga: alis-dwq E1 floor-spike case study (parts 1 & 2).
⚠️ Requires a patched mlx-lm with the glm_moe_dsa indexer fixes (see Correctness). The stock port is incomplete for GLM-5.2; loading there fails or degrades long-context output.
The checkpoint ships GLM-5.2's native MTP (nextn) layer for self-speculative decoding (--mtp — see Native MTP). Backward-compatible: runtimes without MTP support strip the extra layer.
How the weights were made (two orthogonal passes, both chosen by measured A/B):
Anchor-guarded clip-search requantization (alis-dwq clip_quantize): per 128-weight group, candidate clipped grids are accepted only when they lower reconstruction MSE without raising the group's max abs error beyond 1.1× the min-max grid's — preserving the "super weights" that min-max anchors exactly. Source: the public Q8 checkpoint (quasi-continuous; a dequantized nvfp4 sibling as source kills the model via correlated rounding — measured, see the case study). Raw wikitext 4.7109 → 4.4244.
Layerwise DWQ against the 4.5 bpw sibling as teacher (an 8-bit teacher measurably loses at this student capacity — teacher-capacity sweet spot), 45%-ZH calibration, K=6 × 13 rounds with per-round held-out rollback. Wikitext 4.4244 → 3.8985.
quantized from the public Q8 checkpoint (pipenetwork/GLM-5.2-MLX-8bit@531a2ab), not bf16 — disclosed variable vs the from-bf16 siblings
Why this build exists
Find the MLX floor. 2-bit is the container's minimum and group size tops out at 128 → 2.25 bpw effective is the hard floor for the expert bulk. This build tests whether that floor is usable — it is, with the right passes on top.
Method A/Bs on a real 745B student. This repo's family now carries three paired experiments: DWQ teacher precision (4.5 beats 8-bit here), clip acceptance rules (anchor-guarded beats MSE-only, which is fatal), and requant source class (quasi-continuous beats lattice, which is fatal). All documented with numbers in the case study.
Market context: the most aggressive GLM-5.2 quant elsewhere (llama.cpp dynamic "1-bit" UD-IQ1_S) ships at 223 GB ≈ 2.4 bpw effective — every stack converges to this size class on this model. This build undercuts it with measurably recovered quality.
On-disk footprint across GLM-5.2 MLX builds: this experimental 2.32 bpw build (219.8 GB) is the smallest published GLM-5.2; the 2.56 sibling (242 GB) also fits 256 GB; 3.5 bpw (328), mixed-3_6 (360), Q4.8-INF (447) need 512 GB-class machines
Quality
Perplexity: raw 4.711 → clip+DWQ 3.899 (−17.2%) vs 2.56 bpw at 3.698 on wikitext; code 2.272 → 2.105 (−7.3%) vs 2.054
strided PPL (ctx 2048 / stride 1024)
raw E1
this build (clip+DWQ)
2.56 bpw
3.5 bpw
wikitext (prose)
4.711
3.899
3.698
2.777
code
2.272
2.105
2.054
1.835
tulu-3 (mlx_lm.perplexity, 50×2048, seed 123)
3.963
3.608
3.571
3.644
Strided perplexity from a fixed local harness — relative numbers for comparing these builds, not directly comparable to other quantizers' corpora. Sibling columns (and the chart) show each build's current main — both are clip+DWQ retunes as of 2026-07.
CI ±0.02. At 2.32 bpw the task suite sits within ~1 CI of both larger siblings — the bit-rate cost shows up in perplexity, not in these accuracies. (Sibling columns are their current clip+DWQ mains.)
Degeneration probe (greedy 256 tokens/slice, alis-dwq --loop-probe): distinct-4gram EN 0.929 · code 0.901 · ZH 0.984, no cycles detected on any slice — aggregate-score parity can hide loop degeneration (it doubled in a published REAP prune at eval parity), so this build gates on it explicitly.
vs the llama.cpp dynamic quants (same metric class)
Unsloth's GLM-5.2 dynamic GGUFs publish quality as top-1 token agreement against a BF16-or-Q8_0 baseline on a sampled token set. Measuring this build with the same metric (baseline = the Q8_0 checkpoint's stored top-1024 logits; 73,173 tokens across 177 seed-7 calibration samples):
build
size
top-1 vs baseline
llama.cpp UD-IQ1_S ("1-bit")
223 GB
~76.2% (their published figure)
this build (2.32 bpw, clip+DWQ)
219.75 GB
77.56%(our measurement)
llama.cpp UD-IQ2_M ("2-bit")
239 GB
~82% (their published figure)
Smaller than their 1-bit and above its agreement score — while our token set is plausibly harder (45% Chinese, and ZH is the weakest slice at this bit-rate for every build we've measured). Caveat for rigor: same metric class and a baseline within their stated definition ("BF16 or Q8_0"), but different token sets — theirs is a training-corpus subset, ours the calibration mix above — so treat this as indicative, not a shared benchmark. Their 2-bit at +19 GB still leads on this metric, consistent with our own PPL ladder.
Teacher A/B — why the 4.5 bpw teacher ships and the 8-bit one doesn't
Both DWQ arms trained from the same raw student, identical hyperparameters; only the teacher-logit dump differs. Pre-registered prediction: the 4.5 bpw teacher wins at this capacity. It did — on every held-out metric (wikitext +1.25%, code +0.82%, tulu +0.74% for the 8-bit arm), while the 8-bit arm's training valid loss dropped more — the overfit-toward-teacher signature. Rule: judge DWQ teachers on held-out PPL, never on training/valid loss. (Measured on the pre-clip student; the shipped build re-ran the winning arm on the clipped student.)
Teacher A/B: the 8-bit-teacher arm loses every held-out metric vs the 4.5-bpw-teacher arm, and the effect flips sign with student capacity — mid-bit students gain from a sharper teacher, ~2.3–2.6 bpw students lose
Quantization recipe
Mixed-precision recipe: experts 2-bit g128 (the MLX floor, −22.6 GB vs g64), MLA/shared/dense 4-bit, embed/head 6-bit, router bf16, indexer fp16
Component
Bits
Notes
Routed experts (gate/up/down)
2-bit g128
~96% of params — codes decided by anchor-guarded clip-search from the Q8 source
MLA attn · shared experts · dense MLP
4-bit g64
per-token critical path
Token embedding · LM head
6-bit g64
distribution-sensitive
Router (mlp.gate)
bf16
drives discrete top-8 routing
DSA lightning indexer
fp16
drives discrete top-k selection
Native MTP — self-speculative decoding
GLM-5.2 ships a built-in MTP ("nextn") layer predicting token t+2; this build restores it as model.layers.78.* (one extra shard, +4.51 GB). Provenance note: the Q8 source ships no MTP tensors, so the attached layer is the 3-bit-expert sidecar inherited from the 2.56 build's lineage — fine for plain decode (loaders without MTP support drop it in sanitize()), but --mtp speculative gains are unvalidated on this build; expect neutral-at-best single-request speed (acceptance falls as the target quantizes harder). It ships because MTP is exactly lossless.
See the 2.56 card for full MTP long-context notes and integrator details.
Long context & memory
MLA stores a compressed latent KV (~88 KB/token fp16, ~44 KB/token int8; --kv-bits 8 — the patched fork engages int8 on the MLA latent, stock mlx-lm silently ignores it). As on the siblings, the DSA prefill activation — not the KV — binds on a 256 GiB box (~+20 GB per ~30K prompt tokens).
Context-dependent peaks have not been re-measured on this build. Weights-resident peak measured ~215.4 GB — ~23 GB more headroom than the 2.56 build, whose measured curve (249 @8K → 268 @32K → 293 @64K) puts this build's practical prefill ceiling at ~32–40K tokens on a 256 GiB (274.9 GB) machine — an estimate by transposition, not a measurement. For genuinely long context use the 3.5 bpw build on a 512 GiB machine.
Correctness
Produced with the same patched fork as the siblings (glm_moe_dsa fixes: IndexShare top-k reuse; indexer non-interleaved RoPE + eps 1e-6, post-RoPE q matching the HF reference to ~1e-7). Build-specific verification: bit-plan audit (exactly 225 routed-expert modules at 2-bit/g128, router unquantized bf16, tokenizer byte-identical to source, sha 19e77364…), pre-DWQ quality gate, KL-harness consistency check, per-round DWQ rollback gates (12 accepted, 1 late revert — the natural stopping signal), and the degeneration probe above.
Honest caveats:
ZH remains the weakest slice (KL 0.686 vs EN 0.446 after both passes) — the usual low-bit pattern, substantially recovered (raw was 1.154) but not erased.
Instruction-following at this bit-rate: pre-clip raw sanity checks showed occasional English analysis scaffolding instead of the requested output language/format. The shipped build passes the greedy degeneration probe cleanly (above), but a full generation-quality pass is still pending.
From-Q8 provenance (see Metrics) is a disclosed variable vs the from-bf16 siblings — Q8's noise floor is ~38 dB below the 2-bit noise it feeds.
Usage
bash
1# requires mlx-lm with the glm_moe_dsa indexer fixes2mlx_lm.generate --model avlp12/GLM-5.2-Alis-MLX-Dynamic-2.3bpw \3 --prompt "Write a quicksort in Python."45# OpenAI-compatible server6mlx_lm.server --model avlp12/GLM-5.2-Alis-MLX-Dynamic-2.3bpw
78# long context: int8 KV9mlx_lm.generate --model avlp12/GLM-5.2-Alis-MLX-Dynamic-2.3bpw \10 --kv-bits 8 --kv-group-size 64 --quantized-kv-start 4096 --prompt "…"
Generation speed: ~23.3 tok/s single-request greedy (M3 Ultra 512 GB, short prompt, measured on the same-format raw build; clip+DWQ change codes/scales only, not the decode format).
Hardware
Runs in ≤256 GB unified memory (Apple Silicon) with more headroom than any other GLM-5.2 build: a "256 GB" Mac is 256 GiB = 274.9 GB; ~220 GB of MTP-attached weights leave ~55 GB for KV + DSA prefill activation (vs ~33 GB on the 2.56 build).
Credits
Base model: Zhipu / Z.ai — GLM-5.2 (MIT).
MLX & mlx-lm: Apple ml-explore.
Q8 source checkpoint: pipenetwork. Clip-search inspiration: four-over-six (Cook et al., via the humans& NVFP4 RL recipe).
