Mixed-precision MLX quantizations of tencent/Hy3
(295B total / 21B active MoE) for Apple Silicon, sized for 128 / 256 / 512 GiB
unified-memory Macs. One repo, one card — each quantization lives on its own branch.
* single-stream decode on an M3 Ultra, short prompt; longer contexts decode slower.
Final state, all tiers
Everything below is measured, not projected — KL vs the 8-bit reference on a fixed
EN/code/ZH slice, PPL on fixed corpora, and needle retrieval actually run at the stated
context. All builds keep the router in bf16 and expert_bias in fp32.
T128 — 87.6 GB, 128 GiB Macs, experimental. KL 0.550, PPL 5.80 / 2.44 / 5.40.
The only build that fits a 128 GiB Mac's default wired limit; needle verified to
16K fp16 (32K with int8 KV, needs a wired-limit bump). Received both low-bit levers —
sensitivity-placed 3-bit down_proj + Chinese-mixed DWQ (v2) — halving its KL vs v1.
main (T256) — 168.5 GB, 256 GiB Macs, daily driver. KL 0.094, PPL 3.97 / 1.98 /
3.73. Needle verified to 64K fp16 (peak 190 GB, inside the default wired limit);
128K via int8 KV. Uniform 4-bit — already near-optimal for its size (see above).
T512 — 242 GB, 512 GiB Macs, flagship. KL 0.020 (near-lossless), PPL 3.85 / 1.93
/ 3.54, and ~25% faster to decode than the 8-bit. Needle verified to 128K fp16; full
256K fp16 KV fits the default wired limit.
T512REF — 313.5 GB, 512 GiB Macs, reference. Uniform 8-bit; the KL ground truth
and DWQ teacher. Ship only if you want the maximum-quality artifact.
mtp-bf16 — 7.5 GB sidecar + harness. Hy3's MTP head plus a self-speculative
decode loop; measured 1.13× lossless (k=1) on top of any tier.
Which build fits your Mac — weights and measured peak memory vs each machine tier's default wired limit
Which build?
512 GiB Mac → T512. Faster than the 8-bit and statistically near-lossless
(KL 0.020 nats). 128K-token needle retrieval verified at 285 GB peak; 256K fp16
KV fits in the default wired limit (242 + 80 = 322 GB < ~384 GB).
256 GiB Mac → main. 64K-token needle verified at 190 GB peak — just inside the
default ~192 GB wired limit. For 128K, use int8 KV (--kv-bits 8, generate path)
or raise iogpu.wired_limit_mb.
128 GiB Mac → T128, with eyes open: see the experimental warning below.
16K fp16 verified at 93.8 GB peak; 32K with int8 KV peaked at 109 GB, which
requires raising the wired limit on a 128 GB machine.
KL ground truth / archival → T512REF (uniform 8-bit).
Usage
Hy3 (hy_v3) support is not yet in an mlx-lm release
(ml-explore/mlx-lm#1211 is open).
Until it merges, install from the pinned branch (PR #1211 plus an fp32-router
correctness patch, commit 14f7837):
Long context on the smaller tiers: quantize the KV cache in the generate path
(--kv-bits 8 --kv-group-size 64). Note mlx_lm.server currently serves fp16 KV only.
Recipes (verified from each build's config.json, not from intent)
97.0% of Hy3's parameters are the 192 routed experts, so expert bit-width sets the
size; everything else is cheap to keep at high precision. Common to all tiers:
mlp.router.gate — never quantized (bf16). The sigmoid+bias top-8 routing is a
discrete control path; 62M params total, so full precision is free.
expert_bias (e-score correction) kept fp32, verified in the output shards.
Attention bits = min(expert bits + 2, 8).
Per-tier bit allocation by component
tier
routed experts
expert down_proj
attn / shared / dense
embed / lm_head
T128
2-bit g128
3-bit g128 on 12 sensitivity-ranked layers
4-bit g64
6-bit g64
main (T256)
4-bit g64
4-bit g64
6-bit g64
8-bit g64
T512
6-bit g64
6-bit g64
8-bit g64
8-bit g64
T512REF
8-bit g64
8-bit g64
8-bit g64
8-bit g64
T128 (v2) — two byte-neutral quality levers. (1) The 12 down_proj layers upgraded
to 3-bit are chosen by measured sensitivity (each MoE layer's KL when dropped to
2-bit, probed on the 8-bit reference), not an early/late heuristic — the most
sensitive layers turned out to be the middle band (46-65). (2) A DWQ pass (scales/
biases distilled against the 8-bit build's logits) on a 45% Chinese data mix, so
the language-specialized experts actually receive gradient. Same 87.6 GB, same recipe
bits. Net vs the raw quantization: overall KL −42%, ZH −46% (1.55→0.83). The prior v1
(early/late layers + English-only DWQ) reached only ZH 1.43; it is tagged T128-v1.
Quality
KL divergence & top-1 flip rate vs T512REF — measured on a fixed 3,072-token
slice (English / code / Chinese thirds). Disclosure: the reference is the 8-bit
build, not bf16 — the bf16 model (598 GB) does not fit in 512 GB of RAM.
KL divergence vs the 8-bit reference by language slice, log scale
lm-eval (hellaswag / piqa / winogrande, 0-shot, limit 500, same harness across
tiers — absolute numbers are base-style loglikelihood, useful only for tier-vs-tier
comparison): all three tiers are statistically indistinguishable (±2σ), e.g.
piqa acc 0.75–0.77 across every tier. Loglikelihood tasks are insensitive at this
scale; the KL table above is the discriminating metric.
Long-context needle retrieval (needle at 25% depth, greedy): every listed claim
was tested, not extrapolated —
tier
context tested
result
peak memory
T128 (v2)
16K fp16 · 32K int8-KV
PASS · PASS
93.8 · 109.2 GB
main
16K · 64K fp16
PASS · PASS
174.6 · 190.3 GB
T512
64K · 128K fp16
PASS · PASS
263.8 · 285.2 GB
⚠️ T128 is experimental
2.375 bpw on a 295B MoE with 18.9M-parameter experts is aggressive. The v2 build
generates fluent English/Korean/Chinese in under 88 GB and passes 16K needle
retrieval (peak 93.8 GB). Its remaining distribution gap vs the 8-bit reference is
modest overall (KL 0.55) and no longer collapses on Chinese (ZH KL 0.83 vs v1's 1.43,
ZH PPL 5.40 vs v1's 9.01). Still the most aggressive tier — verify on your workload;
for headroom, the 256 GiB main build is far closer to lossless.
Why only T128 got the extra tuning
The two levers that transformed T128 — placing the higher-bit down_proj layers by
measured per-layer sensitivity, and a Chinese-mixed DWQ pass — are 2-bit-specific,
and we verified this rather than assumed it:
Sensitivity shifts with bit-width. Probing each MoE layer's KL when dropped to a
given bit-width, the most fragile layers at 2-bit are the middle band (46–65) but at
4-bit they are the early layers (1–14) — so T128's layer set doesn't transfer. And a
single layer's 4-bit damage is only ~39% of its 2-bit damage: far less to reclaim.
The byte-neutral trick backfires at 4-bit. A mixed down_proj recipe (5-bit on
sensitive layers, 3-bit on robust ones, same total size) improved T128 but measured
worse on main (paired ΔKL −0.013): the 4-bit error surface is flat enough that
demoting robust layers costs more than promoting sensitive ones saves.
DWQ has little to correct at 4-bit. Starting a DWQ pass on main, the initial
validation loss was 0.05 vs T128's ~0.5 — main already sits ~10× closer to the
8-bit reference, and its Chinese KL (0.169) never collapsed the way T128's did (1.55).
So main, T512, and T512REF ship as uniform quantizations: at 4-bit and above the
naive recipe is already near-optimal for its size, and every intervention we measured
was neutral-to-negative. The aggressive tuning is reserved for the tier that needs it.
How it compares to other Hy3 MLX ports
Every public Hy3(-preview) MLX quantization we could find, measured on the same
harness, same machine (M3 Ultra 512GB, wired-limit set, quiet disk): identical
3,072-token KL slice vs our 8-bit reference, identical PPL corpora, same prompt for
decode speed. Sorted by size:
does not load in stock mlx-lm (custom config: no bits in its quantization block)
† KL(8-bit ref ‖ model) on the fixed EN/code/ZH slice — lower is better.
‡ mlx-community's build quantizes Hy3-preview, a different base checkpoint than
the final Hy3; its KL vs our final-Hy3 reference includes base-model drift, so compare
it on PPL. Its uniform 4-bit g64 recipe (8-bit routers) decodes faster than mixed
recipes because its attention path carries fewer bytes per token.
Reading the table:
4-bit class (~161–169 GB): at essentially the same size, main has the lowest
KL of any final-Hy3 build (0.094 vs 0.172 for the imatrix MXFP4 build — ~45% lower)
and the best code/ZH perplexity, at equal speed.
low-bit class: T128 (v2) at 87.6 GB now has the lowest KL of any sub-160 GB
Hy3 build measured (0.550 vs ox-ox's 0.686 at 112.6 GB and unigilby's 0.172 at
161 GB), and it is the only build that fits a 128 GB Mac's default wired limit.
ox-ox spends 25 GB more (8-bit attention) yet scores worse and exceeds the 128 GB
ceiling before the first token, decoding at 8.9 tok/s vs T128's 27. If your Mac has 256 GB, use main instead; if it
has 128 GB, T128 is the build that actually fits.
Routers: ours keep the sigmoid+bias router in bf16 (ox-ox and mlx-community
quantize it to 8-bit; unigilby's config applies its 4-bit default to the router).
With 192 experts and near-tie top-8 margins, full-precision routing is the cheapest
correctness insurance in the recipe (~124 MB total).
Hy3 ships a Multi-Token-Prediction layer (model.layers.80.*, 3.75B params,
DeepSeek-V3-style eh_proj/enorm/hnorm + MoE block) that mlx-lm strips at
conversion. The mtp-bf16 branch preserves it verbatim in bf16 (7.5 GB) plus a
runnable self-speculative-decode harness (mtp_run.py + module + step generators).
Measured on the 8-bit build (M3 Ultra, greedy, 200 tokens, output verified
token-identical to plain decode):
config
tok/s
speedup
accept-len
plain decode
21.4
1.00×
—
MTP k=1 (bf16 draft)
24.1
1.13×
1.65
MTP k=2
22.9
1.07×
2.02
MTP k=3
19.7
0.92×
2.13
Notes: k=1 with the bf16 sidecar is the operating point. A 4-bit draft was
measured too (isolated draft call 2.26 → 1.74 ms, −23%): the saving is real but
amounts to ~1% of loop time (the 47 ms verify forward dominates), while
acceptance drops slightly and consistently across EN/KO/code prompts
(accept-len −0.02…−0.06) — net ≈ −1…−4% vs bf16, tie on code. Use the 4-bit
draft only if you want its 5.4 GB memory saving. The harness uses the MTP's own
KV cache and final-layernorm ("normed hidden") chaining for k>1.
Cross-framework logit parity vs transformers 5.12.1 reference implementation on
tiny-random weights: top-1 agreement 100%, max |Δlogit| ≤ 5e-3 (fp32).
One correctness patch on top of PR #1211: the router matmul runs in fp32,
matching the reference F.linear(hidden.float(), weight.float()) — near-tie
top-8 selection under a sigmoid+bias router is sensitive to bf16 matmul noise.
Recipe verification: quantization blocks parsed from each output config.json;
router dtype and expert_bias fp32 checked in the shards themselves.
Credits
Tencent Hunyuan for Hy3 (Apache 2.0) and
the AngelSlim compression toolkit — the
official quantization path for CUDA deployments (FP8/INT4 for vLLM/SGLang).
These MLX builds are an independent, complementary Apple-Silicon path.
