A quantized build of GLM-5.2 tuned for CPU/GPU hybrid MoE serving on a single GPU that can't hold all 256 experts/layer at higher precision: routed MoE experts are self-quantized to INT4 (compressed-tensors, per-group MSE-calibrated), and the attention path (q_b_proj, o_proj, and the shared-expert MLP in most layers) is swapped to an INT8 mix, freeing enough memory to run a substantially larger KV-cache pool and a weight-staging overlap system without giving up single-stream decode speed.

This is the weight side of a larger effort — see the paired serving-stack forks for the runtime work that makes this checkpoint fast (not the vanilla upstream projects, which lack the fixes below): cfao-new/sglang (branch bisect-no58) and cfao-new/ktransformers (branch kt064-glm52). Those READMEs document the actual measured wins (sparse attention restoration, native fp8 MLA decode, weight-staging overlap, and the bugs fixed along the way); this card focuses on the checkpoint itself.

The INT4 expert quantization was run in-house against the official zai-org/GLM-5.2-FP8 release using a custom MSE grid-search calibrator built on compressed-tensors; it is not a redistribution of a third-party quantization. The INT8 attention/shared-expert tensors are QuantTrio's own public release, mixed in via the module-level scheme their build uses.

A straight INT4 quantization of everything leaves accuracy on the table on a few precision-sensitive attention/expert-selection paths. Swapping just the attention projections and shared-expert MLP to INT8 — the QuantTrio donor set — recovers that margin while keeping the bulk of the parameter count (the routed experts, which dominate total size) at INT4. The net effect versus an all-INT4 build: comparable weight footprint, meaningfully better numerical headroom on the attention path, and — combined with the serving stack's own weight-staging optimizations — enough freed memory to run this model with a much larger KV-cache pool than the unmixed build supports.

Built and validated against the paired sglang + kt-kernel forks linked above. Key serving flags (see the forks' READMEs for the complete, currently-adopted configuration):

--attention-backend nsa            # native sparse attention (this is a trained-in DSA model)
--kv-cache-dtype fp8_e4m3          # native SM90 fp8 MLA decode
--max-total-tokens 524288          # funded by the memory this quantization mix frees up

See the included LICENSE file for the full MIT text and copyright notice.