Uncensored Qwen3.8-27B, published as GGUF
quantizations with the multi-token prediction (MTP) head retained and verified.
Refusal behaviour has been substantially reduced, not eliminated. See Measured
behaviour for the numbers. Capabilities, training data, and architecture are otherwise
unchanged.
MTP tensors verified, not assumed. Abliteration drops the mtp.* tensors: the model
is re-saved through transformers, which does not carry the MTP head, while config.json
still advertises it. They are grafted back from the base checkpoint and every file is
inspected after quantization — see Method and Verification.
Method
Refusal directions removed with Heretic, which
co-minimizes refusal count against KL divergence from the base model. No hand-written
refusal-removal code, no fine-tuning, no additional training data.
Abliteration runs at bf16 (no 4-bit quantization); the resulting LoRA is merged into the
bf16 base, so the published weights are not a quantized round trip.
mtp.* tensors are copied verbatim from the base checkpoint after merging. Abliteration
never touches them — it modifies attn.o_proj and mlp.down_proj in the main stack.
The draft head was trained against the unmodified model, so acceptance rate may fall
slightly. Speculative decoding verifies every token against the target, so output quality
is unaffected.
imatrix is computed directly from the f16, not from an intermediate quantization, so
calibration sees the real weights.
Image input, if the base model ships a vision tower.
MTP Head for llama server
Qwen3.8-27B-Uncensored-draft-Q8_0.gguf
MTP head for llama-server, specifically for the flag --model-draft
The draft head stays at Q8_0 in every configuration. It is small relative to the target, and
quantizing it harder costs draft acceptance rate for almost no disk saving.
Measured on this build, every quant in one session against the same f16 baseline, so the
rows are comparable to each other.
File
PPL (wikitext-2)
vs f16
Qwen3.8-27B-Uncensored-f16.gguf (baseline, not shipped)
7.1557 +/- 0.25104
Qwen3.8-27B-Uncensored-Q5_K_M.gguf
7.1573 +/- 0.25055
+0.0016
Qwen3.8-27B-Uncensored-IQ4_XS.gguf
7.1583 +/- 0.25019
+0.0026
Qwen3.8-27B-Uncensored-Q6_K.gguf
7.1689 +/- 0.25149
+0.0132
Qwen3.8-27B-Uncensored-Q8_0.gguf
7.1764 +/- 0.25195
+0.0207
Qwen3.8-27B-Uncensored-Q4_K_M.gguf
7.1814 +/- 0.25227
+0.0257
Qwen3.8-27B-Uncensored-IQ2_M.gguf
7.8581 +/- 0.27481
+0.7024
Read the error bars before reading the ordering. Every row except IQ2_M sits inside a
span of 0.026 against a standard error of roughly 0.25, so those quants are not separable
from the f16 or from each other, and their ordering here is noise. Do not conclude that Q8_0
is worse than Q5_K_M. The only difference this measurement actually resolves is IQ2_M, which
is about 2.8 standard errors above the baseline.
The noMTP-* twins are not listed because they measure identically to their fused
counterparts. The MTP block is inert during a normal forward pass, which was confirmed here:
fused and noMTP IQ2_M both return 7.8581, and fused and noMTP f16 both return 7.1557.
Corpus is Salesforce/wikitext,
wikitext-2-raw-v1, test-00000-of-00001.parquet, text column joined with \n. This is
the same file used for the importance matrix, described in full below.
Perplexity detects gross quantization damage and nothing else. It does not measure
reasoning, code, multilingual ability, or refusal behaviour. See Caveats that matter.
Importance matrix
Qwen3.8-27B-Uncensored-imatrix.dat is the importance matrix every quantization in this
repo was built with. All twelve of the IQ2_M, IQ4_XS, Q4_K_M, Q5_K_M, Q6_K and
Q8_0 files, fused and noMTP alike, record it in their own metadata. The standalone
draft-Q8_0 head and the vision-f16 projector do not, because neither was built with one.
It is published so the files here can be reproduced, and so you can build quants that are
not in this repo.
text column joined with \n, giving 1,292,013 bytes, md5 d998c24b049cf7c009dbf2672da70b5a
Chunks
200 x 512 tokens
Computed from
the f16 GGUF, not an intermediate quantization
Built with
llama.cpp a94d563ed
Two things to know before you use it:
It is GGUF-format despite the .dat extension (general.type = imatrix). llama.cpp
builds predating GGUF imatrix support will not load it.
It contains no entries for blk.64, the MTP block. llama-imatrix never activates the
draft head during a normal forward pass, so no activations are collected for it. This
matters below.
Provenance is checkable rather than asserted: re-quantizing the f16 to Q4_K_M with this file
reproduces the published Qwen3.8-27B-Uncensored-Q4_K_M.gguf to byte-identical tensors
across all 866 tensors.
Building other low bit quants yourself
The floor here is IQ2_M at 10.6 GB. If you want something smaller, or a type that is not
published, the imatrix above lets you build it.
You will need the f16 GGUF, which is not published here because it is 54 GB. Build it from
the bf16 weights, which are public:
Add --no-mtp to that command for the noMTP variant. The MTP shard is already grafted
into the bf16 repo, so nothing needs restoring first.
The MTP block must be pinned. Because the imatrix has no blk.64 entries, and because
IQ3_XXS, IQ2_XXS, IQ2_S and IQ2_M require per-tensor importance data, a fused low-bit build
without a pin does not merely degrade the draft head. llama-quantize refuses to run at all.
Pinning blk.64 to q8_0 sidesteps the requirement and keeps the draft head intact:
--token-embedding-type q4_K is the largest size lever on this model. llama.cpp force-bumps
token_embd to Q5_K on every IQ2/IQ1 ftype, and at 248320 vocab that is roughly 8 to 10% of
parameters. Do not go below q4_K. Omit the --tensor-type pin for noMTP-* builds, since
there is no block to pin.
Verify afterwards that the block survived, rather than assuming it did:
2-bit warning. IQ2_M is the most degraded file in this repo, and anything you build
below it will be worse. Expect the loss to land hardest on the thing this model is used
for. Behaviour near the old refusal boundary is already its least stable property (see
Caveats that matter), and 2-bit compounds exactly that. Perplexity will tell you the model
is not broken. It will not tell you the refusal boundary still behaves the way it does at
Q6_K, and nothing in this repo measures that at 2-bit.
Third party quants
A third party publishes a range of mixed-precision variants derived from this model:
Not produced by or affiliated with this repo. It is not built by me, I have not verified
its files or its published numbers, and any metrics quoted there were not measured on the
harness used here, so they are not comparable to the perplexity figures in this card. Linked
because people ask for sizes I do not ship, not as an endorsement.
--spec-draft-n-max defaults to 3. Throughput depends on your hardware, so sweep it —
measurements across draft lengths are in
qwen3.8-spec-decode-bench.
Verification
Each artifact was checked post-quantization for MTP tensor survival rather than inferred from
the conversion flag:
This reports metadata keys, declared block_count, and blocks actually present. A fused file
whose present-block count does not exceed its declared count did not retain the MTP block.
File
MTP
blocks
Qwen3.8-27B-Uncensored-f16.gguf
True
65/65
Qwen3.8-27B-Uncensored-noMTP-f16.gguf
False
64/64
Qwen3.8-27B-Uncensored-IQ2_M.gguf
True
65/65
Qwen3.8-27B-Uncensored-noMTP-IQ2_M.gguf
False
64/64
Qwen3.8-27B-Uncensored-IQ4_XS.gguf
True
65/65
Qwen3.8-27B-Uncensored-noMTP-IQ4_XS.gguf
False
64/64
Qwen3.8-27B-Uncensored-Q4_K_M.gguf
True
65/65
Qwen3.8-27B-Uncensored-noMTP-Q4_K_M.gguf
False
64/64
Qwen3.8-27B-Uncensored-Q5_K_M.gguf
True
65/65
Qwen3.8-27B-Uncensored-noMTP-Q5_K_M.gguf
False
64/64
Qwen3.8-27B-Uncensored-Q6_K.gguf
True
65/65
Qwen3.8-27B-Uncensored-noMTP-Q6_K.gguf
False
64/64
Qwen3.8-27B-Uncensored-Q8_0.gguf
True
65/65
Qwen3.8-27B-Uncensored-noMTP-Q8_0.gguf
False
64/64
Measured behaviour
Benchmarked against the unmodified base model on identical settings. The delta is the
figure that matters: it isolates what the weight edit cost.
Task
Base
Uncensored
Δ
MMLU
83.4
83.3
-0.2
ARC-Challenge
58.9
57.7
-1.2
HellaSwag
82.8
82.9
+0.1
Winogrande
76.1
75.3
-0.8
Mean
-0.5
0-shot via lm-evaluation-harness,
bf16, both models scored in the same session. Every delta is within or close to the
reported standard error (MMLU +/- 0.30, ARC +/- 1.44, HellaSwag +/- 0.38, Winogrande +/- 1.21), so
none is clearly separable from run-to-run noise.
These are 0-shot and are not comparable to Qwen's published scores, which use few-shot
prompting. They are directly comparable to each other, which is the point. Note also that
ARC-Challenge is low for a model at this MMLU — the base scores 58.9 under the same
settings, so that is format sensitivity in a reasoning-tuned model, not abliteration
damage.
What the benchmarks do not cover: no generative evaluation (GSM8K, HumanEval), no
math or code, no multilingual, and the harness loads the text stack only — nothing here
measures the vision tower or MTP speculative decoding.
Measurement
Base model
This model
Refusals (100 held-out harmful prompts)
98/100
12/100
KL divergence vs base (first-token)
0
0.1191
Search: 200 Heretic trials, 23 non-dominated points. The published model is the marked row.
refusals
KL divergence
12/100
0.1191
← published
13/100
0.1052
19/100
0.0722
23/100
0.0635
26/100
0.0507
27/100
0.0410
35/100
0.0406
36/100
0.0387
41/100
0.0366
44/100
0.0352
46/100
0.0334
48/100
0.0331
51/100
0.0321
52/100
0.0294
60/100
0.0290
76/100
0.0280
77/100
0.0247
83/100
0.0204
86/100
0.0193
91/100
0.0170
96/100
0.0146
97/100
0.0044
98/100
0.0004
How to read these
Refusal rate is the count of refusals over 100 held-out prompts from
mlabonne/harmful_behaviors
(test split) — explicitly harmful requests, not benign ones. So this number is not an
over-refusal rate: it does not tell you how often the model declines legitimate work. It
tells you how much of the original safety behaviour on harmful requests remains.
KL divergence is measured against the unmodified base model over first-token
distributions, and is the optimizer's proxy for "how much did we damage the model". Lower is
closer to base. It is a proxy, not a capability measurement — a low KL does not certify that
reasoning or coding ability survived, and nothing here does certify that.
The two trade off against each other. Heretic searches a Pareto front between them; the
published point is one choice on that front, not a global optimum.
Caveats that matter
Refusals were measured in non-thinking mode. This model's chat template opens a
<think> block, so the evaluation closes it explicitly to score answers rather than
reasoning traces. With thinking enabled the refusal rate may differ, in either direction.
The measurement is 100 prompts from one dataset. It generalizes to that distribution
of harmful requests and no further. Refusal behaviour on other topics is uncharacterized.
Perplexity is wikitext-2 only (see the Files table). It detects gross quantization
damage. It does not detect capability loss on reasoning, code, or multilingual work.
Quantization compounds everything above. The measurements were taken on the bf16
merge; the files you download are quantized.
Requirements
MTP speculative decoding landed in llama.cpp PR #22673. Builds older than that will load
these files and silently ignore the MTP tensors.
Limitations
Refusals are reduced, not eliminated, and not redirected. This model attempts many requests
the original declines, but a meaningful fraction still get refused — see Measured behaviour.
Behaviour near the old refusal boundary is less stable than the base model.
Lower quants compound that, and IQ2_M compounds it most. Evaluate behaviour on Q6_K or
Q8_0, not on IQ2_M or IQ4_XS. Nothing here measures the refusal boundary at 2-bit.
Capability benchmarks show a 0.5-point mean drop vs base across MMLU, ARC-Challenge,
HellaSwag and Winogrande. See Measured behaviour. No generative, math, code, or
multilingual evaluation was run.
Intended use
Local inference. Not intended for deployment to third parties without your own safety layer.
License
Apache 2.0, inherited from Qwen/Qwen3.8-27B. The base model's license and acceptable use
policy still apply to your use of this derivative.
Speculative decoding, measured on this model
prompt
spec_type
n_max
tok/s
vs baseline
prose
none
-
74.8
1.00x
prose
draft-mtp
1
89.0
1.19x
prose
draft-mtp
2
85.7
1.15x
prose
draft-mtp
3
72.0
0.96x
prose
draft-mtp
4
71.1
0.95x
prose
draft-mtp
5
62.9
0.84x
prose
draft-mtp
6
53.8
0.72x
prose
draft-mtp
7
49.9
0.67x
prose
draft-mtp
8
59.9
0.80x
code
none
-
74.7
1.00x
code
draft-mtp
1
95.4
1.28x
code
draft-mtp
2
92.9
1.24x
code
draft-mtp
3
82.6
1.11x
code
draft-mtp
4
74.9
1.00x
code
draft-mtp
5
67.4
0.90x
code
draft-mtp
6
59.4
0.80x
code
draft-mtp
7
55.6
0.74x
code
draft-mtp
8
70.9
0.95x
chat
none
-
74.7
1.00x
chat
draft-mtp
1
90.6
1.21x
chat
draft-mtp
2
84.2
1.13x
chat
draft-mtp
3
76.1
1.02x
chat
draft-mtp
4
70.4
0.94x
chat
draft-mtp
5
64.3
0.86x
chat
draft-mtp
6
55.2
0.74x
chat
draft-mtp
7
50.2
0.67x
chat
draft-mtp
8
54.1
0.72x
IQ2_M
Measured on this build: NVIDIA H200 NVL, 256 generated tokens, median of 3 repetitions,
n_max swept 1 to 3. The table above dates from the original release and its hardware is
not recorded, so compare the ratios rather than the absolute rates.
prompt
spec_type
n_max
tok/s
vs baseline
prose
none
-
75.4
1.00x
prose
draft-mtp
1
85.2
1.13x
prose
draft-mtp
2
83.2
1.10x
prose
draft-mtp
3
77.4
1.03x
code
none
-
75.5
1.00x
code
draft-mtp
1
95.8
1.27x
code
draft-mtp
2
99.8
1.32x
code
draft-mtp
3
96.0
1.27x
chat
none
-
75.3
1.00x
chat
draft-mtp
1
87.4
1.16x
chat
draft-mtp
2
81.3
1.08x
chat
draft-mtp
3
78.4
1.04x
The MTP head survives 2-bit quantization because it is pinned to q8_0 rather than
quantized with the rest of the stack, so speculative decoding still pays here.
Pairing noMTP-IQ2_M with the published draft-Q8_0 reaches 97.7 tok/s on the prose prompt
at n_max 2, or 1.30x, which beats the fused file on that prompt. The reason is that the
fused MTP head shares the low-bit token_embd (q4_K) and output (Q5_K) tensors with the
main model, while the standalone draft carries its own Q8_0 copies. The split setup needs
13.3 GB of weights against 10.6 GB for the fused file, so it is the better option only if
you have the VRAM to spare.
