This is a calibration-free, safetensors-only expert-pruned checkpoint derived from moonshotai/Kimi-K3. It removes 75% of the routed experts and is intended for vLLM serving and evaluation smoke tests while accurate calibration-based pruning, such as REAP support in llm-compressor, is under development.

This is an experimental prototype, not a REAP-pruned checkpoint. Expect a large quality regression relative to the source model.

Pruning

Source experts per MoE layer: 896
Retained experts per MoE layer: 224 (25% retained, 75% removed)
MoE layers: 92
Ranking: descending signed sum(router.weight[i]), independently per layer
Removed data: pruned expert tensors and their corresponding router rows
Router and expert indices were compacted, and the config was updated to 224 experts
Calibration data: none
Retained expert count is divisible by TP=4 (56 experts per rank)
Tensor payload: 475,131,719,168 bytes (442.50 GiB)
Indexed weights: 126,276 across 96 safetensors shards

pruning_manifest.json records the source hashes, exact per-layer expert remap, pruning method, and output statistics. The checkpoint was validated against every shard keyset and payload size, router and bias permutation, sampled expert tensors in all layers, and auxiliary-file hashes. Each retained 224-expert set is the exact prefix of the corresponding ranking used for mgoin/Kimi-K3-pruned50.

Validated vLLM run

The checkpoint loaded and served successfully with the default vLLM loader (no --load-format) on one node with four NVIDIA GB300 GPUs:

vLLM: 0.26.1rc1.dev142+g30b4e7f47
vLLM commit: 30b4e7f479674a9c4d8889d4857294d3bd5e6849
FlashInfer: 0.6.16rc5
Hardware: 4x NVIDIA GB300, 284208 MiB each
Parallelism: TP=4 on one node
Weight load time: 210.80 seconds
Model memory: 119.06 GiB per GPU
Available KV cache: 132.14 GiB per GPU
KV-cache capacity: 7,615,283 tokens
Steady GPU allocation: approximately 262.90 GiB per GPU

bash

vllm serve mgoin/Kimi-K3-pruned75 \
  --served-model-name kimi-k3-pruned-75pct \
  --host 0.0.0.0 \
  --port 18082 \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --moe-backend auto \
  --gpu-memory-utilization 0.95 \
  --max-model-len 32768 \
  --kv-cache-dtype fp8 \
  --attention-backend FLASHINFER_MLA \
  --attention-config '{"mla_prefill_backend":"FLASHINFER","use_prefill_query_quantization":true}' \
  --enable-prefix-caching \
  --no-enable-flashinfer-autotune \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --max-cudagraph-capture-size 8

The health endpoint returned HTTP 200 and an arithmetic smoke test passed. Kimi K3 latent-MoE tail fusion reported that TP=4 is unsupported and automatically used the default path; serving remained healthy.

GSM8K

The evaluation used vLLM's standalone GSM8K harness against the /v1/completions endpoint with the full test split:

bash

.venv/bin/python tests/evals/gsm8k/gsm8k_eval.py \
  --host http://${HEAD_NODE} \
  --port 18082 \
  --num-questions 1319 \
  --num-shots 5 \
  --max-tokens 256 \
  --temperature 0 \
  --seed 42

Checkpoint	Accuracy	Correct	Invalid	Latency	Questions/s
Kimi-K3-pruned75	34.4200%	454/1319	4/1319	536.326 s	2.459
Kimi-K3-pruned50	68.2335%	900/1319	3/1319	481.863 s	2.737
Source Kimi-K3	95.5269%	1260/1319	1/1319	530.362 s	2.487

The pruned75 checkpoint regressed by 61.1069 percentage points relative to the source model and 33.8135 points relative to pruned50. All runs used the same 1,319-question test set, 5-shot prompt builder, 256-token limit, temperature, seed, vLLM checkout, and GB300 hardware.

License

This derivative is distributed under the upstream Kimi K3 License. See LICENSE in this repository.