Apple Silicon (MLX) mixed-precision quantization of zai-org/GLM-5.2 — a 744B-parameter (~40B active) Mixture-of-Experts model with DeepSeek-V3.2-style MLA + DeepSeek Sparse Attention (DSA, glm_moe_dsa). Quantized to ~2.56 bits/weight so the full model runs in ≤256 GB of unified memory.
⚠️ Requires a patched mlx-lm with the glm_moe_dsa indexer fixes (see Correctness below). The stock port is incomplete for GLM-5.2; loading there fails or degrades long-context output.
New: the checkpoint now ships GLM-5.2's native MTP (nextn) layer for self-speculative decoding (--mtp — see Native MTP). Fully backward-compatible: runtimes without MTP support strip the extra layer and behave exactly as before.
Update 2 (2026-07): anchor-guarded clip + DWQ re-run.main is now rebuilt from a Q8 source with an anchor-guarded clip search (per-group outlier clipping at max-err slack 1.1 that preserves the min-max anchor weights; method in alis-dwq), then DWQ-retuned with the same recipe as before (4.5 bpw-teacher targets, 45%-ZH calibration, upstream flag mlx-lm#1499). Held-out perplexity improves or holds everywhere vs the previous DWQ main: strided wikitext 3.774 → 3.698 (−2.0%), code 2.069 → 2.054, tulu-3 flat (3.564 → 3.571 ±0.027); the clip alone moves the raw pre-DWQ build 4.34 → 4.17. The checkpoint also sheds 4.5 GB: the MTP layer used to ship twice (a module-named copy inside the base shards plus the standalone MTP shard), and now ships once.
KL / top-1 flip vs the 4.5 bpw reference (T=3072)
pre-DWQ
first DWQ (dwq-noclip)
clip+DWQ (main)
EN
0.727 / 24.9%
0.383 / 15.6%
0.464 / 17.3%
code
0.252 / 12.7%
0.193 / 10.2%
0.183 / 10.0%
ZH
0.987 / 35.7%
0.562 / 21.9%
0.610 / 24.8%
overall
0.655 / 24.4%
0.379 / 15.9%
0.419 / 17.4%
How to read this table: the reference is the 4.5 bpw quantized sibling, not ground truth. The first DWQ was derived from the same 4-bit-lattice family as that reference, so it sits unusually close to it; the clip step re-derives every tensor from a Q8 source (removing the lattice-resonance failure mode), which drifts slightly away from this particular reference even as held-out perplexity — the metric that tracks real quality — improves. Judge builds by the PPL numbers. Greedy loop-probe (256 tok × EN/code/ZH): clean, no cycles.
Weights history (retired 2026-08-28): original quant (pre-dwq) · first DWQ (dwq-noclip) were removed from the Hub. Only main (clip + DWQ) still hosts weights.
Metrics
Base model
zai-org/GLM-5.2 (744B total / ~40B active)
Bits/weight
~2.56 (per-tensor mixed)
On-disk size
242.4 GB (47 shards: 46 base + the 4.5 GB native MTP layer, which no longer ships twice)
1M-capable architecture (DSA); machine-limited in practice — ≈26–32K prefill on a 256 GiB box (see Long context & memory)
Speculative decoding
native MTP (nextn layer 78 included)
Why this model
GLM-5.2 is a frontier agentic-coding MoE, but at 744B it is ~1.5 TB in bf16 — out of reach for consumer memory, and existing MLX builds start at ~360 GB (≥4-bit, 512 GB-class machines). This build uses Unsloth-style per-tensor mixed precision: the routed experts (~97% of params) go to 2-bit while the sensitive paths keep higher precision, landing under 256 GB while preserving long-context retrieval and coding quality.
On-disk footprint across GLM-5.2 MLX builds: this 2.56 bpw build (242 GB) is the only one that fits a ≤256 GB machine; golden 328, mixed-3_6 360, Q4.8-INF 447, DQ4plus 465 all need 512 GB-class
Quality
This is the ≤256 GB option — the routed experts are 2-bit, so it is deliberately bit-starved. If you have a 512 GB machine, the 3.5 bpw build is still better on prose/code strided PPL (−25% wikitext, −11% code; both builds are clip+DWQ retunes, the 3.5 bpw sibling against an 8-bit teacher) and runs a full 1M context.
Perplexity: this 2.56 bpw build vs the 3.5 bpw build — 2.56 bpw is 25% higher (worse) on wikitext, 11% on code
Strided perplexity from a fixed local harness — relative numbers for comparing these two builds, not directly comparable to perplexities other quantizers report on different corpora.
Benchmarks
Reproduced with mlx_lm.evaluate (0-shot) and mlx_lm.perplexity (seq 2048, 50 samples, seed 123), against the author's earlier GLM-5.1 quant under the same harness and settings:
Note (2026-07-10): the GLM-5.1 · 2.7 bpw build referenced below (avlp12/GLM-5.1-Alis-MLX-Dynamic-2.7bpw) has been removed from the Hub to reclaim storage; its figures are kept here for reference only.
GLM-5.1 · 2.7 bpw
GLM-5.2 · 2.56 bpw (this)
GLM-5.2 · 3.5 bpw
Perplexity (lower)
4.165
3.571
3.644
HellaSwag (acc_norm)
0.606
0.638
0.626
PIQA (acc)
0.796
0.812
0.838
WinoGrande (acc)
0.660
0.744
0.780
Generation (tok/s)
18.35
22.87
21.29
Perplexity here is on allenai/tulu-3-sft-mixture (the mlx_lm.perplexity default) — a different corpus and method from the wikitext strided figure above, so values are not comparable across the two. Task accuracies use a 500-sample limit (CI ±0.02–0.04). All figures in this column are measured on the current main (clip+DWQ); vs the previous retune (dwq-noclip: 0.652 / 0.808 / 0.780) the task deltas are within combined sampling noise (≤1.3σ at this sample size), consistent with the flat tulu. The 3.5 bpw column is that build's current main. GLM-5.1 is a different (older) base model, so cross-generation gaps reflect both the newer model and quantization.
GLM-5.2 ships a built-in MTP (multi-token-prediction / "nextn") layer that predicts token t+2 from the backbone's hidden state at t — DeepSeek-V3-style. Public MLX builds (including earlier revisions of this one) stripped it at conversion; this build restores it as model.layers.78.* (one extra shard, +4.51 GB; experts 3-bit g64, attention 4-bit, indexer/norms/eh_proj bf16 — measured on the 3.5 bpw sibling: drafter precision does not affect acceptance, bf16 ≡ 3-bit).
Backward compatible: loaders without MTP support drop layer 78 in sanitize() and behave byte-identically to the previous revision. With the fork, --mtp turns it on:
Reference numbers (M3 Ultra 512 GB, single request, greedy): the 3.5 bpw sibling measures ~performance-neutral (20.4 → 20.7 tok/s, accept-len ~2.3) and a 4-bit sibling measured +11% (22.0 → 24.6, accept-len 2.46) — draft acceptance drops as the target is quantized harder, because a noisier target's own outputs are harder for any drafter to predict. On this 2-bit-expert build expect neutral-at-best single-request speed (not yet benchmarked end-to-end here); the layer stays worth shipping because it is exactly lossless, and gains grow with target precision and multi-node serving.
Lossless: at temperature 0 the output is the model's own greedy output; at temperature > 0 drafts are accepted only when they equal the target's sampled token (distribution-exact).
Long context: MTP works past index_topk (2048 tokens — the DSA-sparse regime). Draft acceptance holds there, and the fork's small-L gather path (commit 946c412) keeps the multi-token verify from paying the sparse-mask setup per iteration (before that fix, long-context MTP decoded at 0.26× plain). Measured on a 4-bit sibling at ~2.1K context: --mtp k=2 ≈ 0.85× plain, k=1 ≈ 0.98× — at very long contexts prefer --mtp-num-draft-tokens 1 or plain; the k=2 gain lives in the ≤2048 dense regime.
Memory note: the MTP-capable fork attaches the module at load (+4.5 GB) even without --mtp, trimming the KV+activation budget by that much. To reclaim it, set "num_nextn_predict_layers": 0 in a local config.json copy (or use a runtime without MTP support, which strips the layer). See Long context & memory for the machine-level ceiling.
Notes for other integrators (verified against vLLM's deepseek_mtp semantics): eh_proj concat order is [enorm(embed(t+1)), hnorm(h_t)]; the hidden fed to the MTP is the backbone's pre-final-norm residual stream; for chained drafting feed back the shared_head.norm-normalized hidden (chaining the raw layer output halves chained acceptance — measured 0.27 → 0.75 conditional at position 2).
Long context & memory
GLM-5.2's MLA stores a compressed latent KV (kv_lora 512 + rope 64 per layer), so the KV cache itself is small — ~88 KB/token at fp16, ~44 KB/token with int8. Quantize it with --kv-bits (the patched fork engages int8 on the MLA latent; stock mlx-lm silently ignores it):
On a 256 GiB machine the KV is not the binding constraint — the DSA prefill activation is. Measured peak on this build (M3 Ultra, int8 KV, MTP-attached weights ≈ 242 GB):
prompt (prefill)
peak
fits 256 GiB (274.9 GB)?
8K
~249 GB
✓
26K
~263 GB
✓ (tight, ~12 GB headroom)
32K
~268 GB
✓ (very tight)
64K
~293 GB
✗
128K
~344 GB
✗
The DSA lightning indexer scores every past token per prefill chunk, so the activation peak climbs ~+20 GB per ~30K tokens — independent of KV bits (int8 vs fp16 differed by only ~1 GB at 26K, because the KV is tiny there). So on a 256 GiB box, keep prompts ≤ ~26–32K. --kv-bits 8 still helps where the KV is what grows — long decode/accumulated context, and 512 GiB machines with headroom for longer prefills. For genuinely long (100K–1M) context, use the 3.5 bpw build on a 512 GiB machine.
Correctness (verified vs the HF reference)
GLM-5.2's glm_moe_dsa needed fixes beyond the stock mlx-lm port; this build was produced with a patched fork and validated:
IndexShare — the DSA indexer runs only on "full" layers; "shared" layers reuse its top-k (index_topk_freq=4). The stock port built an indexer on every layer → missing-weights / wrong >2048-token output.
Indexer RoPE/eps — the indexer uses non-interleaved (half-split) RoPE + LayerNorm eps 1e-6, distinct from the interleaved main attention. Post-RoPE q matches the HF reference to ~1e-7. Recorded in config.json (indexer_rope_traditional=false, indexer_norm_eps=1e-6).
Native MTP — layer 78 restored + --mtp self-speculative decoding in the fork; greedy output verified identical to non-speculative decoding (on the 3.5 bpw sibling), chained drafting matches vLLM's deepseek_mtp semantics (normed-hidden chaining).
Validation: full-attention logits match the HF reference to float precision at ≤index_topk context; needle retrieval succeeds through a 7,586-token prompt (sparse-DSA regime); coherent code generation; peak ≤256 GB.
Usage
bash
1# requires mlx-lm with the glm_moe_dsa indexer fixes2mlx_lm.generate --model avlp12/GLM-5.2-Alis-MLX-Dynamic-2.56bpw \3 --prompt "Write a quicksort in Python."45# OpenAI-compatible server6mlx_lm.server --model avlp12/GLM-5.2-Alis-MLX-Dynamic-2.56bpw
78# with native MTP speculative decoding (see the Native MTP section)9mlx_lm.server --model avlp12/GLM-5.2-Alis-MLX-Dynamic-2.56bpw --mtp
Hardware
Runs in ≤256 GB unified memory (Apple Silicon). A "256 GB" Mac is 256 GiB = 274.9 GB (not 256 decimal); the ~242 GB of (MTP-attached) weights leave ~33 GB. But the DSA sparse-attention prefill activation grows with context (measured +~20 GB at 26K, +~50 GB at 64K), so the practical prefill ceiling on a 256 GiB machine is ~26–32K tokens (peak ~263–268 GB) — not the architecture's 1M. For 100K+ context use a 512 GB (512 GiB) machine. See Long context & memory below.
