You can deploy this model and start earning money today!

peasantsmith/
GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF

Name: GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF
Availability: InStock

Views

No views yet

GLM-4.5-Air-PS-IQ2_XXS-MTP GGUF

Compact GGUF quantization of zai-org/GLM-4.5-Air by PeasantSmith, quantized down from the full-size BF16 release. The only known quantized release that keeps the MTP (multi-token prediction) head intact for speculative decoding.

Specs

Format: GGUF

Base model: MoE, 106B total / 12B active params, 128 routed experts + 1 shared, top-8 routing, 46 layers (1 dense + 45 MoE), GQA (96 attention heads / 8 KV heads), 131,072 token context

Size: 45.49 GB / 42.4 GiB, 3.29 bits/param

Source: full-size BF16 GGUF (5 shards, 803 tensors), quantized with llama.cpp (commit c060ca9) using unsloth's published imatrix (502 entries) from the same release

MTP: 1 extra prediction layer embedded (blk.46.nextn.*), not present in other public quants of this model

Quantization Matrix

Source: full-size BF16 GGUF (5 shards, 803 tensors), quantized with llama.cpp (commit c060ca9) using unsloth's published calibration importance matrix (imatrix, 502 entries) from the same full-size release.

Tensor class	Type	Rationale
`blk.1..45.ffn_gate_exps` / `ffn_up_exps`	IQ2_XXS	4096-col, imatrix-covered
`blk.1..45.ffn_down_exps`	Q4_0	1408-col (Q2_K impossible: 1408 % 256 = 128); down-exps precision on residual
`blk.46.*` (MTP block: gate/up/down)	Q4_0	MTP draft head — no imatrix stats, quality immaterial
`blk.46.nextn.*` (MTP head)	Q4_K	embedded MTP, kept reasonably precise
attention / router / output	Q8_0	lowest error per byte
`token_embd` / `output`	Q6_K	quality-critical
Layer 0 (dense)	Q8_0 (default)	no expert tensors

Zero fallbacks; 90 IQ2_XXS tensors. SHA-256: 2f2a8c1234528ec00b0c73dfcb5d33395517e050cbbc7729a2249265bc16305c

Testing Results

Perplexity (512-token context, 67 chunks, calibration text): 5.5835 ± 0.1016. Measured on the calibration text itself, so treat as optimistic relative to held-out data.

Test	Result
Factual QA (capital of France)	correct (Paris)
LRU implementation task	correct: O(1) doubly-linked-list design
Exact-value recall (magic value 9137)	exact

MTP Head

glm4moe.nextn_predict_layers = 1, MTP head embedded in-source at Q4_K (blk.46.nextn.*: embed_tokens, eh_proj, shared_head_head, norms)
Usable via llama.cpp: llama-server --spec-type draft-mtp

Run it

text

`# Standard inference
llama-server -hf peasantsmith/GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF --temp 0.6 --top-p 0.9 -n 4096`

text

`# With the embedded MTP head, for speculative decoding
llama-server -hf peasantsmith/GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF --temp 0.6 --top-p 0.9 -n 4096 --spec-type draft-mtp`

llama-cli works the same way in place of llama-server for a terminal session instead of a server. Needs at least the 42GB file in memory, plus more for context.

Run with ollama

ollama run hf.co/peasantsmith/GLM-4.5-Air-PS-IQ2_XXS-MTP-GGUF:IQ2_XXS

The tag comes from the GGUF filename (last dash-separated segment). The embedded MTP/nextn head is available via llama.cpp (--spec-type draft-mtp); ollama runs the model as a standard MoE.

License

MIT, inherited from the base model.

Integrity

SHA-256: 2f2a8c1234528ec00b0c73dfcb5d33395517e050cbbc7729a2249265bc16305c File size: 45,490,244,672 bytes. Verify after download with sha256sum.

Additional information

The model name PS-IQ2_XXS-MTP is the PeasantSmith release signature. It identifies this quantized release and should not be confused with the upstream model's original naming.