GLM-5.2 SQG W4A8
This is an all-SQG, BF16-source-only GLM-5.2 checkpoint. Routed
expert tensors use independent per-tensor K3/K4 assignments (384
of each per layer); the 380 selected non-routed matrices use SQG
K6. It contains 58,748 SQG matrices and zero MCG marker tensors.
Routed experts use the full-W4A8 endpoint: both h and
SiLU(gate) * up are A8. Dense K6 matrices are calibrated in that
construction context but materialized and served as native SQG
W6A16.
Routed gate/up calibration uses the frozen BMM Law blend
H13 = 0.75 * H13_layer + 0.25 * H13_expert, with routed
applied-router-gate-square weighting and the sealed per-tensor
rate map.
Down projections are encoded from the candidate-conditioned
W4A8 path using the recorded (H, B) cross-term construction.
Hadamard rotations and input-side scales remain on the activation
side so native SQG E4M3 weight labels reach FP8 MMA unchanged.
Release state: owner-directed unvalidated release on main; KLD and runtime acceptance were skipped.
Release regime
- Tensor parallel: 1
- Pipeline parallel: 8, partition
[9, 10, 10, 10, 10, 10, 10, 9]
- Final-logit KLD: not run (owner-directed speed publication)
- Runtime acceptance: not run (owner-directed speed publication)
- Official BF16 revision:
b4734de4facf877f85769a911abafc5283eab3d9
- Calibration/Hessian dataset:
db55cc25ea16c49b458d1a3f1a4661095be96ba4
- Canonical deduplicated calibration view:
canonical-v1
This format requires the custom SQG W4A8 vLLM runtime. It is not a
drop-in Transformers checkpoint. Do not start it as TP8, DCP4, an
SM120-only MCG image, or an A16 fallback. The accepted topology is
PP8/TP1 and the supplied launch artifacts enforce it.
Run
Set VERDICTAI_IMAGE to the immutable image tested for this release:
1export VERDICTAI_IMAGE=verdictai/glm52-sqg-w4a8-sm103@sha256:53641269fccf60a80e2467cc8502dc273fe4d0bfb33bbf2bd894d36262fe08f2
2./serve.sh start
3./serve.sh logs
The API binds to 127.0.0.1:8000 by default. Exact construction,
runtime, acceptance, and dataset identities are recorded in
RELEASE_PROVENANCE.json and the two FULL_SQG_NATIVE_*.json files.
Reproduce the quantization
See
REPRODUCE_SQG_W4A8.md for the
complete BF16-source capture, H13/H2/B calibration, per-tensor
K3/K4 routed encoding, dense-K6 encoding, materialization, and
publication procedure. Exact sealed scripts and source hashes are
in the linked calibration dataset under
reproduction/ and
derived/lineage/reproducibility/.
HESSIAN_DATASET_LAYOUT.md identifies
the ready-to-use H13 files, canonical activation views, dense-H
inputs, and intentionally repeated archival paths so users do not
download all 3.1 TB unnecessarily.
Credits and lineage
- Z.ai for
GLM-5.2.
- Luke Alonso for QSRT/KQuant and
B12X work. The QSRT/KQuant
source history from initial commit
3acea57fb5a82b3b848504bdc87953420f531f39
through audited snapshot 104dd9233f850a3955f4991bea68b07dd34deeb8 is
authored by Luke. The repository is not linked because no working public URL
was verified.
- Brandon Music for the GLM-specific
corpus, Hessians, encoder adaptations, model assembly, validation, and
publication.
- turboderp for the reused
ExLlamaV3 v0.0.43
encoder, trellis-reference, compatibility, and codebook components present
in the published reproduction source. EXL3's documented research lineage
includes QTIP and
QuIP#; those papers are not authors of
the custom SQG method.
- Special thanks to local-inference-lab
for its vLLM fork, Blackwell
runtime/image work, testing, and review.
The exact source and calibration artifacts are the linked, revision-pinned
Hessian datasets above.
License
This repository does not relicense GLM-5.2 or third-party runtime
code. The derived checkpoint remains subject to upstream GLM-5.2
terms; see
LICENSE. Reused ExLlamaV3 components remain subject to its
MIT license.
The audited QSRT/KQuant snapshot contains no top-level license declaration, so
this card does not infer one; resolve that source-license notice before
redistributing those source files independently.
