Views
No views yet
| Base model | zai-org/GLM-5 |
| Format | GGUF |
| Total params | 381B |
| Active / token | — |
| Experts / layer | — |
| Layers | — |
| Hidden size | — |
| Context | — |
| On-disk size | 130 GB |
| Variant | Format | Link |
|---|---|---|
GLM-5-381B | BF16 | link |
GLM-5-381B-GGUF-BF16 | GGUF | link |
GLM-5-381B-GGUF-IQ2_M (this) | GGUF | link |
GLM-5-381B-GGUF-IQ2_XXS | GGUF | link |
GLM-5-381B-GGUF-Q3_K_M | GGUF | link |
GLM-5-381B-W3A16 | W3A16 | link |
glm5-reap-observations | BF16 | link |
| Category | Q3_K_M (170GB) | UD-IQ2_M (121GB) | UD-IQ2_XXS (97GB) |
|---|---|---|---|
| Math (GSM8K) | 8/10 (80%) | 6/10 (60%) | 2/10 (20%) |
| Reasoning (BBH) | 8/10 (80%) | 7/10 (70%) | 4/10 (40%) |
| Coding (HumanEval) | 9/10 (90%) | 8/10 (80%) | 7/10 (70%) |
| Agentic (SWE-bench) | 10/10 (100%) | 10/10 (100%) | 10/10 (100%) |
| Terminal-bench | 9/10 (90%) | 9/10 (90%) | 10/10 (100%) |
| Overall | 44/50 (88%) | 40/50 (80%) | 33/50 (66%) |
| Property | Value |
|---|---|
| Base model | zai-org/GLM-5 (744B, 256 routed experts) |
| Pruning | REAP saliency pruning, 50% expert removal (256 -> 128 experts) |
| Quantization | Dynamic IQ2_M with imatrix (2.72 BPW) |
| Size | ~121 GB |
| Architecture | GlmMoeDsaForCausalLM (MLA + MoE + DSA) |
| Context | 202,752 tokens |
| Active params | ~20B per token (8 of 128 experts) |
| Component | Quant Type | Rationale |
|---|---|---|
output.weight | Q5_K | Critical for logit quality |
token_embd.weight | Q4_K | Important for input representation |
| Dense FFN (first 3 layers) | Q5_K | No MoE; all traffic flows through |
| MLA attention projections | Q4_K | Coherence-critical |
MLA key projection (attn_k_b) | Q5_K | Higher precision for keys |
DSA indexer (indexer.*) | Q5_K | Critical for attention routing |
Shared experts (ffn_*_shexp) | Q4_K-Q5_K | Always active |
| Last MoE layer (blk.77) | Q3_K | Final representation |
| Routed MoE experts (bulk) | IQ2_M | Main savings; imatrix-calibrated |
1huggingface-cli download 0xSero/GLM-5-381B-GGUF-IQ2_M --local-dir ./model
2
3./llama-server \
4 --model ./model/GLM-5-REAP-50pct-UD-IQ2_M.gguf \
5 --ctx-size 8192 \
6 --n-gpu-layers 99 \
7 --port 8080 \
8 --reasoning-budget 2048| Variant | BPW | Size | Parse Rate | Repo |
|---|---|---|---|---|
| BF16 | 16.00 | 711 GB | N/A | BF16-GGUF |
| Q3_K_M | 3.82 | 170 GB | 88% | Q3_K_M-GGUF |
| UD-IQ2_M (this) | 2.72 | 121 GB | 80% | this repo |
| UD-IQ2_XXS | 2.19 | 97 GB | 66% | UD-IQ2_XXS-GGUF |
1@misc{lasby2025reap,
2 title = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression},
3 author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa},
4 year = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}
5}