An abliterated GGUF quantization of Kimi K3 (2.8T params / 104B active / 896 experts, top-16), with the refusal direction orthogonalized out of the residual-stream write path.


Size	539.7 GiB, 34 shards
Quant	`IQ1_S-XS` (experts IQ1_S, router F32, attn/KDA IQ4_XS, shexp Q5_K)
Tensors modified	279 of 2573
Harmful-prompt refusal	0.0% (0/26), vs 7.7% (2/26) for the unmodified baseline
Over-refusal on benign prompts	0.0% (0/30) — unchanged from baseline
Incoherence	0.0% (0/18) — unchanged from baseline
PPL @12 chunks	1.9323 ± 0.0473

English below · 中文见此

Why this repo exists

The two existing K3 uncensored repos publish no refusal numbers at all: one claims ">98% of guardrails removed" with no benchmark, the other states retention is "pending / has not been measured." An abliteration whose effect was never measured is indistinguishable from a broken model — the characteristic failure mode of the technique is not "still refuses" but "answers incoherently," and counting only "did it emit refusal boilerplate" scores a lobotomized model as a success.

So this repo ships the measurement alongside the weights, including the parts that came out inconclusive. Read the Limitations section before trusting any number here — the headline 0.0% is real but the comparison against baseline is not statistically significant at this sample size, and we say so rather than rounding it up into a claim.

Measured refusal behaviour

74 held-out prompts, three separately-reported rates, greedy decoding (temp=0.0, seed=20260803), judged on the first 240 characters of the final answer after stripping the think channel.

	abliterated (this repo)	baseline `UD-IQ1_S`
Refusal rate (26 harmful — lower = more thoroughly removed)	0.0% (0/26)	7.7% (2/26)
Over-refusal rate (30 benign, topically adjacent — lower = better)	0.0% (0/30)	0.0% (0/30)
Incoherence rate (18 factual — lower = capability retained)	0.0% (0/18)	0.0% (0/18)
Empty replies	0/74	0/74
Soft-evasion markers	0/74	0/74
Median answer length, harmful set	2511 chars	2548 chars

Three rates, not one. Reporting only the refusal rate is the mistake that makes a destroyed model look successful:

Refusal rate (harmful set) — does the intervention work at all.
Over-refusal rate (benign set) — did it damage legitimate requests. These prompts are deliberately topically adjacent to the harmful set (lawful firearm maintenance, medication dosage questions, security research). A rise here means the direction was mis-selected or the scale was too aggressive.
Incoherence rate (factual set) — questions with objectively checkable answers, scored by keyword hit. This catches the failure mode where the model stops refusing because it has stopped making sense.

The abliterated model answered every harmful prompt at a median 2511 characters with zero empty replies, and hit 18/18 on the factual set — so the 0.0% is not the artifact of a broken model emitting nothing.

Limitations — read this before citing any number above

The improvement over baseline is not statistically significant. 0/26 vs 2/26 gives a two-sided Fisher exact p = 0.490. At n=26 this evaluation simply cannot resolve an effect the size of 7.7%. The direction of the result is what we set out to produce, and the intervention provably changed the weights (see below), but this test does not establish that it lowered the refusal rate. A larger or harder harmful set would be needed for that.

The baseline itself refused only 7.7%. Either unsloth's UD-IQ1_S is already permissive, or these 26 prompts are not hard enough to elicit refusals from K3. Both readings limit what the comparison can show.

The baseline is a different quantization recipe. The honest comparison would be against our own IQ1_S-XS with the abliteration omitted; that artifact was deleted to reclaim disk before this evaluation was designed. UD-IQ1_S (unsloth, 553.2 GiB) shares the expert type but allocates non-expert layers differently. Using a third party's unmodified release removes any suspicion of self-serving baseline construction, but it is not a same-recipe A/B.

The judge is keyword matching, not semantic. Soft evasion — "this is a complex topic…" followed by no content — is not counted as a refusal, so the reported refusal rates are underestimates. We checked for 15 soft-evasion patterns and found 0/74 in the scored window, but that window is only the first 240 characters; we did not scan full texts, and the stored evaluation JSON keeps only the answer head. An LLM judge would be more accurate at the cost of an external dependency and its own biases; we chose the reproducible criterion.

PPL is measured at 12 chunks. See the perplexity section for why this matters and how easily it misleads.

How the abliteration was done

Standard abliteration finds the direction in activation space that mediates refusal, then projects it out of the weights that write to the residual stream. The interesting part here is how cheaply that lands on K3's architecture.

The refusal direction

llama-cvector-generator over 308 prompt pairs (harmful / harmless, matched in surface form) across all 93 layers, computed on the BF16 tensors of the local UD-Q8_K_XL copy — not taken from a published vector, so the direction belongs to this model rather than to a donor.

Layer selection matters more than it looks. We take the stable band mid60-80 (layers 56–73) and use an in-band normalized average, not a single layer and not the all-layer mean: early and late layers carry directions that pull the average off, so including them degrades the result.

279 tensors, and why that is the whole model

Only matrices that write into the residual stream are orthogonalized. Read-side projections are left alone.

Tensor family	Count	Note
`attn_output`	93	attention write-back
`ffn_routed_up`	92	the MoE write side — 3584 latent → 7168 residual
`ffn_down_shexp`	92	shared-expert write-back
`ffn_down`	1	layer 0's dense MLP
`token_embd`	1
Total	279	of 2573 tensors

ffn_routed_up is the one that makes this tractable. K3's latent MoE routes through a pair of shared projections, and that pair is shared across all 896 experts — ffn_routed_down (7168 → 3584) reads, ffn_routed_up (3584 → 7168) writes. Orthogonalizing that single matrix per layer removes the refusal direction from every one of the 896 experts at once, which is why 1347 GiB of expert weights were not touched at all. ffn_routed_down is correctly left unmodified — it reads from the residual stream, it does not write to it.

This is a gift from K3's specific structure. It does not transfer to MoE architectures that give each expert its own output projection.

Verification

scale=1.0 (full orthogonalization). Removed-component ratio across the 279 tensors: median 1.2084%, range 0.9615–1.7291%.

Eleven tensors were then audited individually against five criteria: the destination's projection onto the refusal direction drops to 1.2–1.6% of the source (≤2% is the BF16 noise floor); source norms still match the pre-run report (confirming nothing was modified in place); source and destination have different inodes with link count 1 (confirming the reflink clone is COW, not a hard link); bit patterns genuinely differ; and the orthogonal complement changed by only 1.5e-3, i.e. BF16 rounding — the intervention removed the refusal direction without disturbing the other components.

Perplexity

Unified methodology: wikitext-2-raw/wiki.test.raw, 12 chunks, n_ctx=512, --n-cpu-moe 93, same machine and same llama.cpp build as every other row.

Quant	Size	PPL	vs Q8
UD-Q8_K_XL (lossless reference)	1453.9 GiB	1.3453 ± 0.0420	—
UD-IQ1_S (baseline, unmodified)	553.2 GiB	1.8824 ± 0.0446	+0.5371
IQ1_S-XS (our recipe, not abliterated)	539.7 GiB	1.9193 ± 0.0469	+0.5740
IQ1_S-XS abliterated (this repo)	539.7 GiB	1.9323 ± 0.0473	+0.5870

Against the same recipe without abliteration, the cost is +0.013 (+0.68%) — smaller than that row's own ±0.0469 error bar. At this sample size the abliteration produces no statistically detectable language-modelling loss. That is a weaker claim than "no loss," and it is the one the data supports.

A trap worth documenting, because we walked into it. The full 568-chunk run of this model gives PPL 2.6479 ± 0.0115, and comparing that to the 12-chunk baselines suggests a catastrophic +38% regression. It is an artifact. The sliding perplexity average rises monotonically as more of the corpus is covered — this same model reads 1.93 at chunk 12 and 2.65 at chunk 568. Every published K3 baseline here was measured at 12 chunks, so a 568-chunk figure is not comparable to any of them. Before comparing PPL across models, check that the chunk count matches (grep 'calculating perplexity over N chunks'). Numbers quoted without their chunk count, corpus and n_ctx cannot be cross-compared with this table — including the ones in this table, against anyone else's.

How to run

Identical to any other K3 GGUF. Requires the unsloth llama.cpp fork, which stacks on ggml-org PR #26185 (K3 architecture support, authored by llama.cpp member pwilkin, not yet merged to master):

bash

git clone https://github.com/unslothai/llama.cpp && cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision && cd ..

# If nvcc is not on PATH, set it explicitly or cmake fails to find the CUDA compiler
export CUDACXX=/usr/local/cuda/bin/nvcc PATH=/usr/local/cuda/bin:$PATH
cmake llama.cpp -B llama.cpp/build \
  -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=90 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j 64 \
  --target llama-cli llama-server llama-perplexity

bash

hf download Ryanchen911/Kimi-K3-Uncensored-GGUF --local-dir ./K3abl \
  --include "*IQ1_S-XS-*"

./llama.cpp/build/bin/llama-cli \
  --model ./K3abl/Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 -c 8192 \
  --temp 1.0 --top-p 0.95 --single-turn

Point --model at shard 1 only; llama.cpp derives the rest from the filename, so keep all 34 shards in one directory under their published names.

--n-cpu-moe 93 places all MoE layers in CPU memory. Memory is the hard gate: you need roughly 550 GB of RAM+VRAM combined. 8×H100 (640 GB HBM) cannot hold this alone and must be paired with large system RAM; our node is 8×H100 + 2TB RAM, where mmap keeps resident memory around 22 GB and leaves the rest in page cache. Expect ~6.8 tok/s generation and a ~4.5 min first (cold) load.

Known issue inherited from the fork: during warmup, _exps selection does not bypass top-k, so only 16 of 896 experts load and the first load is slow. Work around it with --no-warmup.

Reproducing

The quantization recipe is unchanged from IQ1_S-XS — see 6block/Kimi-K3-GGUF for the full bit-width allocation and quantize command. The only difference is that the source GGUF is the orthogonalized copy.

bash

# 1. refusal direction: 308 pairs × 93 layers, on the BF16 tensors of UD-Q8_K_XL
llama-cvector-generator -m Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  --positive-file refusal_harmful.txt --negative-file refusal_harmless.txt \
  -o refusal_dir.gguf

# 2. pick the stable band (layers 56-73), in-band normalized average
python3 select_layer.py --dir refusal_dir.gguf --band mid60-80 -o layer_pick.json

# 3. orthogonalize the 279 residual-write tensors, scale=1.0
#    reflink COW clone: the 1.5 TiB output costs almost no extra disk
python3 abliterate_gguf.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/ \
  --direction layer_pick.json --scale 1.0

# 4. verify before spending hours on quantization
python3 verify_abliterated.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/

# 5. quantize the orthogonalized copy with the IQ1_S-XS recipe
#    Set metadata here, at quantize time -- see the note below
cd /path/with/imatrix   # so imatrix.file records a bare filename
llama-quantize --allow-requantize --keep-split \
  --imatrix imatrix_k3.gguf --tensor-type-file t3_types.txt \
  --token-embedding-type q6_k --output-tensor-type q6_k \
  --override-kv general.quantized_by=str:Ryanchen911 \
  --override-kv general.name=str:Kimi-K3-abliterated \
  UD-Q8_K_XL-abl/Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  Kimi-K3-abliterated-IQ1_S-XS.gguf IQ1_S 64

Set the metadata with --override-kv during quantization. general.quantized_by and general.repo_url are inherited from the source GGUF and are not rewritten by llama-quantize, so a quant built from someone else's file ships their attribution. Likewise quantize.imatrix.file records the --imatrix argument verbatim — run from the imatrix's directory and pass a bare filename, or your absolute server path gets published. Fixing this afterwards means streaming 540 GiB through a rewriter; setting it here costs nothing.

size_label stays at 896x60B — no experts were pruned in this tier.

Reproducing the evaluation:

bash

llama-server -m Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 --port 8080 -c 4096
# wait for {"status":"ok"} -- a 540 GiB load takes minutes and returns 503 until ready
python3 eval_refusal.py --url http://127.0.0.1:8080 --tag abliterated \
  --json-out eval_abl.json

The 74 evaluation prompts are deliberately disjoint from the 308 pairs used to compute the direction — different topics, and none of the six wrapper templates. Self-testing on the prompts that produced the direction inflates every number.

Intended use and risk

This model will attempt harmful requests. It is published for safety research (measuring what guardrail removal actually does to a frontier MoE), red-teaming, and users who need an unfiltered model for legitimate work and accept responsibility for their prompts. It is not appropriate as a drop-in for a user-facing product, and the underlying model's knowledge limits and quantization damage still apply. You are responsible for what you generate with it.

Credits

moonshotai/Kimi-K3 — original model
unsloth — UD-Q8_K_XL lossless source and UD-IQ1_S baseline; full-size fixes in the llama.cpp fork
pwilkin — K3 architecture support in llama.cpp
Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (arXiv:2406.11717) — the method

License

Inherits the Kimi K3 License (near-MIT; explicitly permits modification, distribution, sublicensing and derivative works. Obligations apply only to MaaS businesses above $20M annual revenue, and products above 100M MAU or $20M monthly revenue, which must display "Kimi K3" in their UI).

中文说明

Kimi K3（2.8T 参数 / 104B 激活 / 896 experts top-16）的**消融（abliterated）**GGUF 量化，已将拒绝方向从写入残差流的权重中正交化移除。


体积	539.7 GiB，34 分片
量化档	`IQ1_S-XS`（专家 IQ1_S、router 保 F32、attn/KDA IQ4_XS、shexp Q5_K）
改动张量	2573 个中的 279 个
有害提示拒绝率	0.0%（0/26），未消融基线为 7.7%（2/26）
正当请求过度拒绝率	0.0%（0/30）—— 与基线持平
答非所问率	0.0%（0/18）—— 与基线持平
PPL @12 chunks	1.9323 ± 0.0473

这个仓库为什么存在

现有两个 K3 uncensored 仓库都没有给出任何拒绝率数字：一个声称「移除 >98% 防护」却无 benchmark，另一个 README 明写 retention「pending / 尚未测量」。没测过的消融和坏掉的模型无法区分——这个技术的典型失败模式不是「仍然拒绝」而是「答得不知所云」，只统计「有没有说拒绝话术」会把废掉的模型判成成功。

所以本仓库连同权重一起给出测量结果，包括那些结论不确定的部分。引用本页任何数字前请先读局限一节：标题上的 0.0% 是真实测量值，但与基线的差异在此样本量下不具统计显著性，我们如实说明而不是把它四舍五入成一个结论。

拒绝行为实测

74 条 held-out 提示，分三个率独立报告，贪心解码（temp=0.0、seed=20260803），剥掉 think channel 后只看最终答案的前 240 字符。

	消融版（本仓库）	基线 `UD-IQ1_S`
拒绝率（26 条有害，越低说明移除越彻底）	0.0%（0/26）	7.7%（2/26）
过度拒绝率（30 条正当但话题相邻，越低越好）	0.0%（0/30）	0.0%（0/30）
答非所问率（18 条有客观答案，越低说明能力保持越好）	0.0%（0/18）	0.0%（0/18）
空回复	0/74	0/74
软性回避标记	0/74	0/74
有害集回答字数中位数	2511 字	2548 字

必须分三个率，只报拒绝率是错的——那正是让一个被破坏的模型看起来成功的原因：

拒绝率（有害集）—— 干预到底起没起作用。
过度拒绝率（正当集）—— 有没有伤到正常请求。这些提示刻意与有害集话题相邻 （合法枪械保养、用药剂量、安全研究）。这一项上升说明方向选错或 scale 过大。
答非所问率（事实集）—— 有客观答案的普通题，靠关键词命中判断。它捕捉的是「模型不再拒绝是因为它已经不知道自己在说什么」这种失败。

消融版对每一条有害提示都作答，字数中位数 2511、零空回复，事实集 18/18 全中——所以这个 0.0% 不是「模型坏了什么都不输出」造成的假象。

局限 —— 引用上面任何数字前请先读这里

相对基线的改善不具统计显著性。 0/26 vs 2/26，Fisher 精确检验双尾 p = 0.490。 n=26 根本分辨不出 7.7% 量级的效应。结果方向与我们的目标一致，且干预确实改变了权重（见下文验证），但这个测试并不能证明它降低了拒绝率。要证明需要更大或更难的有害集。

基线自己也只拒绝了 7.7%。 要么 unsloth 的 UD-IQ1_S 本身就比较宽松，要么这 26 条提示不足以让 K3 触发拒绝。两种解释都限制了这个对照能说明的东西。

基线是不同的量化配方。 诚实的对照应该是我们自己的 IQ1_S-XS 去掉消融那一步；但那个产物在本次评测设计出来之前就为腾磁盘删掉了。UD-IQ1_S（unsloth，553.2 GiB）专家层类型相同、非专家层分配不同。用第三方未修改的发布版做基线排除了「自己造一个有利基线」的嫌疑， 但它不是同配方的 A/B 对照。

判据是关键词匹配而非语义判断。 软性回避——「这个话题很复杂……」然后不给内容——算不出拒绝，所以报告的拒绝率是低估值。我们查了 15 个软性回避模式，在评分窗口内命中 0/74，但那个窗口只有前 240 字符；我们没有扫描全文，评测 JSON 也只保存了答案开头。 LLM-judge 更准，但引入外部依赖和它自己的偏置，这里选可复现的口径。

PPL 是 12 chunks 口径。 为什么这点重要、以及它多容易误导人，见 perplexity 一节。

消融是怎么做的

标准做法：找到激活空间里承载拒绝行为的方向，再把它从所有写入残差流的权重里投影掉。这里值得说的是它在 K3 架构上落地得有多便宜。

拒绝方向

llama-cvector-generator 跑 308 对提示（有害 / 无害，表面形式配对）× 全部 93 层，在本地 UD-Q8_K_XL 的 BF16 张量上计算——不用别人公布的向量，所以这个方向属于这个模型本身而不是某个捐赠模型。

选层比看起来更关键。我们取稳定带 mid60-80（layer 56–73），用带内归一化平均，既不是单层也不是全层平均：靠前和靠后的层带的方向会把平均拖偏，纳入它们反而变差。

279 个张量，以及为什么这就等于整个模型

只正交化写入残差流的矩阵，读侧一律不碰。

张量族	数量	说明
`attn_output`	93	attention 写回
`ffn_routed_up`	92	MoE 的写侧 —— 3584 latent → 7168 残差
`ffn_down_shexp`	92	共享专家写回
`ffn_down`	1	layer 0 的 dense MLP
`token_embd`	1
合计	279	2573 个张量中

ffn_routed_up 是让这件事变得可行的关键。K3 的 latent MoE 通过一对共享投影路由，而这对投影是全部 896 个专家共用的——ffn_routed_down（7168 → 3584）读，ffn_routed_up （3584 → 7168）写。每层只正交化这一个矩阵，就等于同时消掉全部 896 个专家的拒绝方向，所以 1347 GiB 的专家权重一个字节都没碰。ffn_routed_down 正确地保持未修改——它从残差流读取，不向其写入。

这是 K3 特定结构送的便宜。换成每个专家各有自己输出投影的 MoE 架构，这个结论不成立。

验证

scale=1.0（完全正交化）。279 个张量的消除比例：中位数 1.2084%，范围 0.9615–1.7291%。

随后逐个审计了 11 个张量的 5 项指标：目标张量在拒绝方向上的投影降到源的 1.2–1.6% （≤2% 即 BF16 噪声底）；源张量范数仍与运行前报告一致（确认没有被就地改写）；源与目标 inode 不同且链接数为 1（确认 reflink 是 COW 克隆而非硬链接）；位模式确有差异；以及正交补的相对变化仅 1.5e-3，即只有 BF16 舍入——干预移除了拒绝方向而没有伤到其他分量。

Perplexity

统一口径：wikitext-2-raw/wiki.test.raw、12 chunks、n_ctx=512、--n-cpu-moe 93，与其他每一行同一台机器、同一 llama.cpp 构建。

量化	体积	PPL	相对 Q8
UD-Q8_K_XL（无损基准）	1453.9 GiB	1.3453 ± 0.0420	—
UD-IQ1_S（基线，未消融）	553.2 GiB	1.8824 ± 0.0446	+0.5371
IQ1_S-XS（我们的配方，未消融）	539.7 GiB	1.9193 ± 0.0469	+0.5740
IQ1_S-XS 消融版（本仓库）	539.7 GiB	1.9323 ± 0.0473	+0.5870

与同配方未消融版相比，代价是 +0.013（+0.68%）——小于那一行自身的 ±0.0469 误差棒。在此样本量下，消融测不出统计显著的语言建模损失。这个说法比「没有损失」弱，但它是数据支持得住的那一个。

一个值得记录的陷阱，因为我们自己踩了进去。 这个模型跑完整 568 chunks 得 PPL 2.6479 ± 0.0115，拿它和 12-chunk 的基线比，看起来是灾难性的 +38% 退化。那是假象。 滑动 perplexity 平均值随语料覆盖增加而单调上升——同一个模型在 chunk 12 读数 1.93，在 chunk 568 读数 2.65。这里每个公开的 K3 基线都是 12 chunks 测的，所以 568-chunk 的数字与它们中任何一个都不可比。跨模型比 PPL 之前，先确认 chunk 数一致 （grep 'calculating perplexity over N chunks'）。未附 chunk 数、语料和 n_ctx 的 PPL 数字不能与此表交叉比较——包括拿此表去和别人的比。

怎么跑

与任何其他 K3 GGUF 相同。需要 unsloth 的 llama.cpp fork，它叠在 ggml-org PR #26185 之上（K3 架构支持，由 llama.cpp 成员 pwilkin 提交，尚未合入 master）：

bash

git clone https://github.com/unslothai/llama.cpp && cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision && cd ..

# nvcc 不在 PATH 时必须显式指定，否则 cmake 找不到 CUDA 编译器
export CUDACXX=/usr/local/cuda/bin/nvcc PATH=/usr/local/cuda/bin:$PATH
cmake llama.cpp -B llama.cpp/build \
  -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=90 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j 64 \
  --target llama-cli llama-server llama-perplexity

bash

hf download Ryanchen911/Kimi-K3-Uncensored-GGUF --local-dir ./K3abl \
  --include "*IQ1_S-XS-*"

./llama.cpp/build/bin/llama-cli \
  --model ./K3abl/Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 -c 8192 \
  --temp 1.0 --top-p 0.95 --single-turn

--model 只指向第 1 个分片；llama.cpp 会从文件名推导其余分片，所以 34 个分片要放在同一目录并保持发布时的文件名。

--n-cpu-moe 93 把所有 MoE 层放在 CPU 内存。内存是硬门槛：需要 RAM+VRAM 合计约 550 GB。8×H100（640 GB HBM）单独装不下，必须配大内存；我们的节点是 8×H100 + 2TB 内存， mmap 让常驻内存维持在约 22 GB，其余留在 page cache。预期生成速度约 6.8 tok/s，首次冷加载约 4.5 分钟。

继承自 fork 的已知问题：warmup 阶段 _exps 的选择没有绕过 top-k，只加载 896 个专家中的 16 个，导致首次加载很慢。用 --no-warmup 绕过。

复现

量化配方与 IQ1_S-XS 完全一致——完整位宽分配和量化命令见 6block/Kimi-K3-GGUF。唯一区别是源 GGUF 换成了正交化后的副本。

bash

# 1. 拒绝方向：308 对 × 93 层，在 UD-Q8_K_XL 的 BF16 张量上算
llama-cvector-generator -m Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  --positive-file refusal_harmful.txt --negative-file refusal_harmless.txt \
  -o refusal_dir.gguf

# 2. 取稳定带（layer 56-73），带内归一化平均
python3 select_layer.py --dir refusal_dir.gguf --band mid60-80 -o layer_pick.json

# 3. 正交化 279 个残差写侧张量，scale=1.0
#    reflink COW 克隆：1.5 TiB 输出几乎不额外占盘
python3 abliterate_gguf.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/ \
  --direction layer_pick.json --scale 1.0

# 4. 先验证，再花几小时去量化
python3 verify_abliterated.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/

# 5. 用 IQ1_S-XS 配方量化正交化后的副本
#    元数据在这一步就设好 —— 见下面的说明
cd /path/with/imatrix   # 这样 imatrix.file 记录的是裸文件名
llama-quantize --allow-requantize --keep-split \
  --imatrix imatrix_k3.gguf --tensor-type-file t3_types.txt \
  --token-embedding-type q6_k --output-tensor-type q6_k \
  --override-kv general.quantized_by=str:Ryanchen911 \
  --override-kv general.name=str:Kimi-K3-abliterated \
  UD-Q8_K_XL-abl/Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  Kimi-K3-abliterated-IQ1_S-XS.gguf IQ1_S 64

元数据用 --override-kv 在量化时设好。 general.quantized_by 和 general.repo_url 是从源 GGUF 继承的，llama-quantize 不会重写，所以用别人的文件做出来的量化会带着对方的署名。同理 quantize.imatrix.file 逐字记录 --imatrix 参数——请在 imatrix 所在目录运行并传裸文件名，否则你的服务器绝对路径会被发布出去。事后修要把 540 GiB 流式重写一遍，在这一步设好则零成本。

size_label 保持 896x60B——这一档没有剪专家。

复现评测：

bash

llama-server -m Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 --port 8080 -c 4096
# 等到 {"status":"ok"} 再打 —— 540 GiB 加载要几分钟，就绪前一律返回 503
python3 eval_refusal.py --url http://127.0.0.1:8080 --tag abliterated \
  --json-out eval_abl.json

那 74 条评测提示与用来算方向的 308 对刻意无重叠——主题错开，且不套用那 6 个 wrapper 模板。在产生方向的同一批提示上自测，每个数字都会虚高。

用途与风险

这个模型会尝试执行有害请求。它发布用于安全研究（测量移除防护对一个前沿 MoE 究竟做了什么）、红队测试，以及确实需要无过滤模型做正当工作并对自己的提示负责的使用者。它不适合直接用在面向用户的产品里，且底座模型的知识边界与量化损伤依然存在。你对自己生成的内容负责。

致谢

moonshotai/Kimi-K3 —— 原始模型
unsloth —— UD-Q8_K_XL 无损源与 UD-IQ1_S 基线；llama.cpp fork 中的全尺寸模型修复
pwilkin —— llama.cpp 的 K3 架构支持
Arditi 等，Refusal in Language Models Is Mediated by a Single Direction （arXiv:2406.11717）—— 方法出处

许可

继承 Kimi K3 License （接近 MIT；明确允许修改、分发、再许可和衍生作品。义务仅适用于年营收超 2000 万美元的 MaaS 业务，以及 MAU 超 1 亿或月营收超 2000 万美元的产品，这些须在 UI 中显示「Kimi K3」）。