Routed experts tensors use the IQ1_S_R4 quant (1.5 BPW)
Indexer tensors use either the Q6_0 quant (6.5 BPW) or the Q8_0 quant (8.5 BPW)
All other tensors use the Q6_0 quant (6.5 BPW)

Coupled with the recent enhancements:

MTP support with e.g. --spec-type mtp:n_max=4,p_min=0.0 (#1890)
graph parallel support with -sm graph (#1821)
DSA support with -dsa -fidx (#2045, #2098, #2109, and many others)

it should run at decent speed as well, with very little slowdown at long context.

(Note: For now, -sm graph and quantize KV cache e.g. -ctk q8_0 do not work together with -dsa -fidx. As a fun exercise, you can ask this quant to get them to work.)

Versions

There are 2 versions:

GLM-5.2-GGUF-1.630bpw.gguf - Made with the imatrix from unsloth (thanks!)
GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf - Made with the imatrix from muzzy (thanks!)

Comparison:

version	imatrix	indexer	ppl
GLM-5.2-GGUF-1.630bpw.gguf	unsloth	Q6_0	6.1409 +/- 0.03770
(not uploaded)	unsloth	Q8_0	6.1409 +/- 0.03770
GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf	muzzy	Q8_0	5.7371 +/- 0.03465

The PPL results were gathered using llama-perplexity with -f wiki.test.raw -c 512.

Recommendation: GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf - The imatrix from muzzy, which took him 5 days to generate, is just amazing. At very low bitrate, imatrix plays an even more important role to keep the quant coherent. From testing, this quant works well enough for agentic coding.

Size

Size from llama-server output:

llm_load_print_meta: model size       = 142.952 GiB (1.630 BPW)
llm_load_print_meta: repeating layers = 141.512 GiB (1.618 BPW, 751.427 B parameters)

Buffer size with -cmoe --no-mmap:

llm_load_tensors:        CPU buffer size = 130712.34 MiB
llm_load_tensors:      CUDA0 buffer size = 13724.12 MiB

Recipe

# Attention
blk\..*\.attn_k_b\.weight=q6_0
blk\..*\.attn_v_b\.weight=q6_0

blk\..*\.attn_kv_a_mqa\.weight=q6_0
blk\..*\.attn_q_a\.weight=q6_0
blk\..*\.attn_q_b\.weight=q6_0
blk\..*\.attn_output\.weight=q6_0

# First 3 Dense Layers
blk\..*\.ffn_down\.weight=q6_0
blk\..*\.ffn_(gate|up)\.weight=q6_0

# Shared Expert Layers
blk\..*\.ffn_down_shexp\.weight=q6_0
blk\..*\.ffn_(gate|up)_shexp\.weight=q6_0

# Routed Experts Layers
blk\..*\.ffn_(up|gate|down)_exps\.weight=iq1_s_r4

# Indexer
blk\..*\.indexer\.proj\.weight=q8_0
blk\..*\.indexer\.attn_k\.weight=q8_0
blk\..*\.indexer\.attn_q_b\.weight=q8_0

# NextN MTP Layer
blk\..*\.nextn\.embed_tokens\.weight=q6_0
blk\..*\.nextn\.shared_head_head\.weight=q6_0
blk\..*\.nextn\.eh_proj\.weight=q6_0

# Non-Repeating Layers
token_embd\.weight=q6_0
output\.weight=q6_0

Flags

Example that uses DSA with IndexShare:

--no-mmap -ngl 99 --fit \
-mla 1 -dsa -fidx \
-c 102400 \
-b 2048 -ub 2048 -wgt 1 \
-cram 0 -muge -cuda graphs=1 \
--jinja --parallel-tool-calls \
--chat-template-kwargs {"reasoning_effort": "high"} \
--spec-type mtp:n_max=4,p_min=0.5

For the very desperate with only 64 GiB of RAM and a GPU, can try to rely on mmap by adding --defer-experts --prefetch-experts --no-warmup and removing --no-mmap -muge, as well as setting the environment variable GGML_CUDA_NO_PINNED=1. From https://github.com/ikawrakow/ik_llama.cpp/pull/2101, this can increase PP by more than 2x! (result varies based on disk speed)

Comparison with other quants

quant	size	ppl
GLM-5.2-GGUF-1.630bpw-muzzy-imatrix.gguf	142.952 GiB (1.630 BPW)	5.7371 +/- 0.03465
IQ1_KT from muzzy/GLM-5.2-GGUF	169.190 GiB (1.928 BPW)	4.4567 +/- 0.02620
UD-IQ1_S from unsloth/GLM-5.2-GGUF	201.823 GiB (2.300 BPW)	4.5930 +/- 0.02686

The IQ1_KT from https://huggingface.co/muzzy/GLM-5.2-GGUF, using the same recipe as the smol-IQ1_KT from https://huggingface.co/ubergarm/GLM-5.1-GGUF, is about 26 GiB larger, and has a much better PPL. Strongly recommended.

On the other hand, the UD-IQ1_S from https://huggingface.co/unsloth/GLM-5.2-GGUF is about 59 GiB larger than this quant, and its PPL is still worse than the IQ1_KT from muzzy. Unless one is stuck with the mainline llama.cpp, there is no good reason to use this unsloth quant.

sokann/GLM-5.2-GGUF-1.630bpw

GLM-5.2-GGUF-1.630bpw

Versions

Size

Flags

Comparison with other quants

sokann/
GLM-5.2-GGUF-1.630bpw