Every one of the 82,432 routed experts is present — no experts are dropped. The GGUF loads without model-specific patches on an existing Kimi-K3-capable llama.cpp build.

The exact method used to create this model will be released soon. This card documents what the model is, how to run it, and how it measures — not how it was built.

Benchmarks

Benchmark	K3 IQ1_S GGUF	Full K3	Notes
HumanEval	94.5% (155/164)	94.5%	1:1 match
MMLU	79.49%	85%	Within expected IQ1_S gap
AIME	92.5%	96%
GSM8K	95%	95%

Metric	Value
Size	330,167,807,328 bytes / 307.49 GiB (9 shards)
vs. smallest published K3 quant	44% smaller (594 GB → 330 GB)
Weights resident	316,036 MiB, measured on 8× A100-80GB
Speed (llama.cpp, single-stream)	10.8 tok/s
Speed (3× DGX Spark TP3, SparkInfer)	12.5492 tok/s (K=8 profile, 112/112 accepted drafts)
WikiText-2 PPL	5.3610 ± 0.02917 (canonical `wiki.test.raw`, all 568 chunks, `-c 512`)
Full-MXFP4 top-1 agreement	73.741 ± 0.243% (32-window mixed-domain paired protocol)
Context	Architecture limit 1,048,576; 3× Spark TP3 practical ceiling ~8K for full resident weights (~113 GiB/rank)

1. SparkInfer TP3/TP4 — Optimal Path (Blackwell)

Current measured throughput: 12.5468 and 12.5516 tok/s, averaging 12.5492 tok/s around a matched 6.4930 baseline, with 112/112 accepted drafts and identical token IDs. This is the K=8 repeat-heavy structured profile from the SparkInfer TP3 recipe, verified on real DGX Spark (GB10) hardware.

SparkInfer is optimized for all Blackwell chips (H200, GB10/DGX Spark, and other GB10-based systems). The gittensor-ai-lab/sparkinfer-k3 repo runs on 8× H200 as the reference configuration; DGX Spark (GB10) has a reduced footprint (~113 GiB/rank under 121 GiB).

Runtime	Hardware	Measured decode
SparkInfer TP3	3× DGX Spark (GB10)	12.5492 tok/s (K=8 profile, 112/112 drafts)
SparkInfer TP4	4× DGX Spark (GB10)	7.90 tok/s
llama.cpp	8× A100-80GB	10.8 tok/s
vLLM TP3 + DSpark	3× H200	52.454 tok/s (coding)

Why MY SparkInfer Patch Is Optimal

Correctness verified 2026-08-11: The distributed (TP3/TP4) generation path had a bug in kimi_k3_dist_rank.cpp: it never set the GGUF capability flags that gate loading, so shared experts, routed-expert normalization, and MLA attention gate were silently skipped on every token — real weights present, never computed. MY fix adds the same capability probe every other code path already had.
Footprint: ~113 GiB/rank fits comfortably under GB10's 121 GiB.
Current measurements (corrected engine):
- Load time: ~5m45s/rank (was ~30-60 min pre-patch 0013)
- SPARKINFER_K3_KDA_FUSE=0: +20% over MY own baseline
- K=4 speculative decoding candidate: 10.3002 and 11.3231 tok/s (10.8117 avg), +68.2% over matched 6.4272 baseline, 95/95 accepted drafts
- K=8 repeat-heavy profile: 12.5468 and 12.5516 tok/s (12.5492 avg) around 6.4930 baseline, 112/112 accepted drafts, identical token IDs
Not valid for non-Blackwell chips: Tested on physical fleets, hard memory ceiling on other hardware.

Full launch procedure (see kimi-k3-neuron-tp3-dgxspark-recipe):

3 or 4 DGX Spark (GB10) nodes, each with ≥320 GB free NVMe
Pull model with hf_xet transport
Apply verified 28-patch chain through 0026
Build SparkInfer with MY patch series
Copy built dist/ tree to every rank node
Launch rank 0 (coordinator) with sparkinfer --model ./Kimi-K3-UD-IQ1_S --tp 3 --rank 0 --listen --ctx-len 8192 --jdba-band
Launch rank 1 and rank 2 (workers) with --coord <rank0-host>:9000 --jdba-band
Verify all ranks connected, then query

Shards load from Kimi-K3-UD-IQ1_S-00001-of-00009.gguf (all 9 parts required).

Speculative Decoding (Experimental, Opt-In)

Patches 0021–0026 add n-gram/prompt-lookup speculative decoding, match-confidence gating, recursive-majority continuation, and distributed LM-head banding. On real TP3 hardware, 64-token generations show:

Code/structured: +27% with --spec-draft 4
Literal repetition: +83% with --spec-draft 4
Freeform prose: −3.7%
Mean across all three: +34.5%

2. llama.cpp — Fallback (Any Chip)

Requires a llama.cpp build with Kimi-K3 support — upstream PR #26185 or the Unsloth K3 fork.

bash

llama-server \
  -m Kimi-K3-UD-IQ1_S-00001-of-00009.gguf \
  -ngl 99 --jinja \
  -c 8192 --parallel 1 \
  -ctk f16 -ctv f16 -fa on \
  --temp 1.0 --top-p 0.95

3. vLLM TP3 + DSpark — Last Resort (Non-Blackwell)

Runtime	Hardware	Measured decode	Recipe
vLLM TP3 + DSpark	3× H200	52.454 tok/s (coding)	`kimi-k3-neuron-tp3-vllm-recipe`

Download

This repository is Xet-backed. Use the hf_xet transport — chunk-deduplicated and parallel:

bash

pip install -U "huggingface_hub[hf_xet]"
hf auth login

export HF_XET_HIGH_PERFORMANCE=1
hf download vcruz305/Kimi-K3-GGUF --local-dir ./Kimi-K3-UD-IQ1_S

Model shards and template only:

bash

hf download vcruz305/Kimi-K3-GGUF --local-dir ./Kimi-K3-UD-IQ1_S \
   --include "Kimi-K3-UD-IQ1_S-*.gguf" --include "k3_chat_template.jinja"

All 9 shards are required; llama.cpp loads them automatically from Kimi-K3-UD-IQ1_S-00001-of-00009.gguf.

Two things learned moving this artifact around:

Xet dedup is per chunk, not per file — when shard 00001 was rewritten to embed the chat template, the 39.4 GB re-upload transferred 395 kB.
Advertised bandwidth is meaningless: a host advertising 538 Mbps sustained 7,828 Mbps on real Xet transfers.

Status

Experimental, and it works. It loads, reasons in K3's native format, and scores 94.5% on HumanEval — measured on the full 164 problems, not a subset.

Changelog

2026-08-12: Renamed GGUF shards to Kimi-K3-UD-IQ1_S-*-of-00009.gguf (server-side pointer rename; no weight re-upload).
2026-08-11: Correctness fix applied and verified in SparkInfer TP3 engine — kimi_k3_dist_rank.cpp now sets GGUF capability flags that gate loading of shared experts, routed-expert normalization, and MLA attention gate. Pre-fix measurements were invalid (engine skipped real computation).
Pre-2026-08-11: Historical speed tables deprecated — measured on engine skipping computation.
Initial release: 330 GB IQ1_S GGUF with all 82,432 experts present, verified loading on Unsloth llama.cpp fork.