Full Kimi K3 in GGUF is ~1.5 TB. This is the same de-risked checkpoint requantized to Q2_K so it fits a single 8×B200 node with room for KV — with the refusal surface already reduced at the weight level in the parent.
The single-node GGUF footprint is the product; the refusal behaviour change is inherited from the parent and is the contract.
Specifications
Architecture
Kimi K3 LatentMoE + KDA · general.architecture = kimi-k3
Requantization to all-Q2_K · built to PR #26185's KV contract
Experts
896 of 896 routed retained — no pruning
On-disk
~940 GiB (1,009 GB) · 38 shards · 2,573 tensors
Context
8,192 validated · higher untested
Serve shape
1× node · 8× B200 · -ngl 99 -nr
Status
EXPERIMENTAL · free, ungated
What "Q2_K" costs here
The experts were already MXFP4 4-bit QAT in the source checkpoint. This is therefore a
requantization of already-quantized weights, not a quantization of BF16. That is a different
and harsher operation than a normal Q2_K build, and its quality relative to the MXFP4 parent
has not been measured.
Blackfrost's own quant ladder for K3 starts at IQ4_XS for exactly this reason. This Q2_K exists to prove single-node GGUF viability, not as a recommended quality point.
Full-precision and higher-quality builds
This Q2_K is the openly-distributed variant. As set out above, it is a requantization of
already-4-bit-QAT experts and is not the quality point Blackfrost recommends for production work.
Full-precision and higher-quality builds of this checkpoint are available for purchase under a
separate licence agreement, including:
Build
What it is
Lossless MXFP4
Routed experts bit-exact from the parent packs — no requantization at all
Coding-calibrated REAP-320
320 of 896 experts, fits a single 8× RTX 6000 node
Custom
Other expert budgets, non-uniform keep sets, or calibration against your own workload and threat model
Access to these builds is granted on purchase. Their repositories are gated, and approval
follows a completed licence agreement — the gate is the transaction. Requesting access to a
premium repository without a licence in place will not be approved.
Purchase link coming soon. Until then, @Blackfrost_AI DMs are
the fastest route to a human.
Refusal-direction de-risk at the weight level (in parent) · Q2_K requantization
Not applied
Expert pruning · additional SFT/DPO
Format
GGUF · llama.cpp PR #26185 KV contract
On refusal behaviour: this checkpoint inherits the parent's deliberately reduced refusal surface. It is a Blackfrost de-risked model. Do not evaluate or rate-limit it as if it were a safety-stock derivative of upstream Kimi K3.
healthy in ~120 s, ~956 GiB across 8 GPUs at -c 8192
Decode
~16.3 tok/s single-stream
Termination
5/5 clean at temperature 1.0 / top_p 0.95
Quality — not yet measured
Benchmark
Parent (MXFP4)
This (Q2_K)
Retention
pending
—
—
—%
Requantizing already-QAT experts is a capability trade, and a card that ships a Q2_K of a 4-bit-QAT parent without publishing what it cost is asking the reader to take the trade on faith. Harness, conditions and retention figures will be stated here — including any benchmark where retention is poor.
⚠️ Sampling — read before filing a bug
Do not use greedy decoding. At temperature 0 / top_k 1 the model fails to terminate: it emits its answer, re-opens a response segment and restates it on an exact 40-token cycle, never producing an end-of-generation token.
Follow Moonshot's published guidance:
{ "temperature": 1.0, "top_p": 0.95 }
Measured on this file, five seeds each, identical prompt:
sampler
clean terminations
temperature 1.0, top_p 0.95
5 / 5 — Moonshot's spec, recommended
temperature 0.6, top_p 0.95
4 / 5
temperature 0, top_k 1
0 / 5 — loops indefinitely
temperature 0, top_k 1, repeat_penalty 1.1
stops — use if you need determinism
This is greedy-decoding degeneration, not a defect in the quantization. Moonshot's card specifies temperature = 1.0; generation_config.json sets eos_token_id: 163586 (<|end_of_msg|>). Greedy was never a supported operating point for K3.
-nr / --no-repack is strongly recommended. Repack is on by default, runs single-threaded (100% of one core, 0% GPU) and allocates a second full copy of the weights.
Point -m at shard 00001 — llama.cpp resolves the remaining 37 automatically.
max_tokens. Thinking is always on; the answer lands in content, the chain of thought in reasoning_content. Budget generously or content comes back empty with finish_reason: length.
Integrity. Verify shard count and byte totals after download before attributing a load failure to the weights.
Benign warning at load
load: special_eom_id is not in special_eog_ids - the tokenizer config may be incorrect
Ignore it.llama-vocab.cpp emits the warning and inserts the token on the next line — it is self-healing. This file's vocab metadata is correct: [BOS] 163584 · [EOS] 163585 · <|end_of_msg|> 163586 · [EOT] 163593. Structural markers are <|open|> 163587 · <|close|> 163588 · <|sep|> 163589.
File naming
Shards carry the legacy KIMI-K3-MXP4-DERISKED-Q2_K-* prefix, which contains a historic MXP4 typo. Filenames are deliberately not renamed so existing download paths and scripts keep working.
⚠️ Security: structural markers in untrusted input
K3 builds prompts in Python via encoding_k3.py::build_chat_segments, where each segment carries its own allow_special flag (default False). A Jinja template cannot express that, and llama.cpp tokenizes the rendered prompt in a single pass with parse_special=true.
Consequence: a literal <|end_of_msg|> in user content becomes control token 163586 in llama.cpp, where Moonshot's own tokenizer keeps it as ordinary text. Untrusted input can forge chat structure.
This is a general llama.cpp property, not specific to K3 or this quant. If you feed untrusted text to this model, strip or escape these four markers first:<|open|> · <|sep|> · <|close|> · <|end_of_msg|>
Disclaimer
Refusal behaviour in this checkpoint has been deliberately modified at the weight level. It is not a safety-stock model and must not be deployed, marketed, or evaluated as one.
No warranty of any kind. Provided "as is", without warranty express or implied, including fitness for a particular purpose. Nothing here guarantees that any given input will be accepted or refused, that any capability is retained, or that any category of output is unreachable.
Measurements describe what was measured. Throughput, termination and retention figures reflect specific harnesses under specific conditions. They are not safety proofs and do not generalise to multimodal, tool-use, long-context or multi-turn adversarial settings.
Modification by a recipient voids this characterization. Blackfrost's obligations attach at the point of release. Any further ablation, fine-tuning, merging, quantization or alteration by a recipient produces an artifact Blackfrost has not evaluated and does not stand behind — responsibility for that artifact transfers entirely to whoever produced it.
Operator-owned policy. Open weights mean the operator sets and enforces policy. Deploy only in controlled environments with access control, independent logging and review. Do not deploy where refusal behaviour equivalent to upstream Kimi K3 is assumed or required.
Access & licensing
This repository is free and ungated — no request, no review, no purchase. It is published openly for the community to evaluate. The full-precision builds listed above are the paid tier; those repositories are gated and their gates open on a completed licence agreement.
Base licence:Kimi K3 — Moonshot AI's terms apply to this derivative and travel with it.
Redistribution: do not redistribute weights outside your grant.
Evaluation recommendation: should not be evaluated by processes that assume refusal behaviour equivalent to the parent.
Ask us about other quant points, or calibration against your own threat model.
