This is a community quantized derivative. Credit for the base weights belongs to Cerebras (REAP prune) and Z.ai (original GLM-4.7-Flash). Please cite / attribute the upstream models when using this checkpoint.

Lineage

text

zai-org/GLM-4.7-Flash
        │
        │  Cerebras REAP — uniform 25% expert prune
        │  (arxiv:2510.13999)
        ▼
cerebras/GLM-4.7-Flash-REAP-23B-A3B   (~23B-A3B MoE, BF16)
        │
        │  ModelOpt PTQ — NVFP4 on routed experts
        │  (attention / gates / shared experts / embeds kept high-prec)
        ▼
dmnsh/GLM-4.7-Flash-REAP-23B-A3B-NVFP4   (this repo)

Field	Value
Immediate base	`cerebras/GLM-4.7-Flash-REAP-23B-A3B`
Upstream ancestor	`zai-org/GLM-4.7-Flash`
Quantization tool	NVIDIA ModelOpt (`hf_ptq`)
Format	NVFP4 (group size 16)
Calibration data	NVIDIA Nemotron SFT / science / math / coding / agentic mixes (ModelOpt default calib set)
License	MIT (inherits from GLM-4.7-Flash / REAP card)

Quantization recipe (what changed)

From hf_quant_config.json (quant_algo: NVFP4, group_size: 16):

Component	Format
Routed MoE experts	NVFP4 (E2M1 + FP8 scales, block size 16)
Attention (`self_attn*`)	BF16 (excluded)
Router / `mlp.gate`	BF16 (excluded)
Shared experts	BF16 (excluded)
First dense layer (`layers.0*`)	BF16 (excluded)
Embeddings + `lm_head`	BF16 (excluded)
KV cache	not quantized in this export (`kv_cache_quant_algo: null`)

Note: ModelOpt calibration initially enabled FP8 KV scales; the published export has KV cache quantization disabled (GB10 / Spark-friendly: serve with --kv-cache-dtype auto).

Intended use

Same as the REAP MoE base, with much lower memory footprint on Blackwell GPUs (e.g. DGX Spark GB10). NVFP4 weight kernels need Blackwell.

Serve with vLLM

Validated on DGX Spark with vllm/vllm-openai:nightly-aarch64:

bash

vllm serve dmnsh/GLM-4.7-Flash-REAP-23B-A3B-NVFP4 \
  --quantization modelopt_fp4 \
  --moe-backend marlin \
  --enforce-eager \
  --kv-cache-dtype auto \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.161 \
  --served-model-name GLM-4.7-Flash-REAP-23B-A3B-NVFP4

Notes:

--moe-backend marlin was required for this MoE NVFP4 checkpoint on the tested nightly.
--enforce-eager and --kv-cache-dtype auto match the GB10 smoke/bench config (FP8 KV disabled).

DGX Spark speedups (BF16 → NVFP4)

vLLM on GB10, max-model-len=4096, concurrency 1.

Metric	BF16	NVFP4	Change
Memory (GiB)	50.78	23.67	−53%
Prefill TTFT (ms)	240.6	123.3	−49% (2.0× faster)
Throughput (tok/s)	24.5	28.5	+16% (1.16×)

Files to ignore in this local folder

If uploading from a local checkout that still has working artifacts, do not upload:

config.json.pre_kv_patch
hf_quant_config.json.pre_kv_patch
.quant_summary.txt

Disclaimer

This checkpoint is provided as-is for experimentation. Quantization can change model quality; run your own evals before production use. Not affiliated with Cerebras, Z.ai, or NVIDIA beyond use of open ModelOpt tooling.