SuperGLM-5.2 v2 W4/W8 overlay — TP4/K5/200K on 4× DGX Spark
This repository distributes our 62 derived W8 overlay shards, not a full checkpoint. Users obtain the pinned upstream packed base separately and assemble the checkpoint locally. The overlay package is approximately 7.13 GB; it avoids duplicating more than 400 GB of unchanged base weights.
Our derived changes: 124 source tensors converted to 372 W8A16 group-128 tensor entries in 62 overlay shards.
Unchanged routed experts remain W4A16; unchanged ordinary linears and the matching 256-expert native MTP remain W8A16.
The final checkpoint is a SuperGLM v2 W4/W8 derivative. QuantTrio is the upstream packed base, not the model identity.
No full base checkpoint is included.
Install and verify
Agent-directed path
An infrastructure agent can start from this page. It must read AGENT_INSTALL.md, then the linked GitHub AGENTS.md. The packaged agent-bootstrap.sh clones the pinned installation repository and enters its guarded prepare/launch workflow.
bash agent-bootstrap.sh --prepare
The script deliberately stops for real cluster discovery when no cluster configuration exists; it never copies private topology values from the reference deployment.
The complete scripts, runtime recipe, benchmarks, and independent verifier are at:
The installer downloads the pinned base directly from QuantTrio, creates a hard-linked local output tree, applies this repository’s overlays and index, and runs the independent verifier. The base and output directories must share a filesystem.
Users can instead reconstruct the overlays directly from the pinned SuperGLM v2 source with ./scripts/build-model.sh.
Cold server-side prefill measured 663.18 tok/s at 5K and 622.43 tok/s at 50K. Identical warm repeats reached 97.73% and 99.812% prefix-cache hits; cached values are not presented as cold prefill throughput.
The package includes provenance and source-evidence hashes. Run the verifier after assembly; do not serve an unverified tree.
Behavior measurement
An internal 32-prompt semantic suite recorded 32 substantive assists, zero refusals, zero garbled outputs, and zero errors through both the direct API and a minimal Hermes profile with thinking off. Raw high-risk prompts and generated harmful text are intentionally not distributed. Results are measurements, not warranties.
Security and use
The reference vLLM server has no application authentication. Keep it on a trusted private network and add authentication, TLS, rate limits, ACLs, and tool sandboxing before sharing it. Generated text is untrusted data.
This is an experimental research and systems-engineering release. Operators are responsible for lawful use, access control, outputs, and compliance with upstream licenses. Read TERMS_OF_USE.md, RESPONSIBLE_USE.md, and LEGAL_NOTICE.md.
Credits
Built from Z.ai GLM-5.2, Jiunsong SuperGLM v2, and QuantTrio’s packed base, with runtime work based on Tony D’Angelo’s DGX Spark recipe, eugr’s build harness, vLLM, and the sparse-MLA contributors credited in the GitHub repository. This project is unofficial and is not endorsed by those parties.
