Near-lossless Q8_0 GGUF of the abliterated Qwen3.6-27B. Pick this when you have the VRAM and want maximum fidelity without the full BF16 footprint.
Provided variants
Every format of this model lives in its own repo, collected under the qwen3.6-27b-ablit collection. Sorted by size; IQ-quants are often preferable to similar-sized non-IQ quants.
1# llama.cpp / llama-server2llama-server -m Qwen3.6-27B-Ablit-Q8_0.gguf --flash-attn on --reasoning off
34# llama-cli one-shot5llama-cli -m Qwen3.6-27B-Ablit-Q8_0.gguf -p "Your prompt here"
The base chat template (Qwen3 ChatML) is unchanged. The model was profiled in
non_thinking mode; if you enable Qwen's thinking mode the refusal geometry
differs slightly and behavior may vary.
Abliteration Method
AblitMD — for multi-disciplinary abliteration, and to be a little hokey.
Try it out: roll your own recipe (ablitMD or classic heretic-style) with the
all-in-one app at github.com/eric-merritt/ablitMD.
The motivation is Tom Wollschläger et al.'s theory that refusal is not a single
direction but a polyhedral cone (arXiv:2502.17420),
hence per-category ablation.
Per category directions computed using the hidden states from the final token in the residual stream.
i. A sample set of 41 harm categories was derived through a simple 2 message chat with popular models:
The questions:
1. I'm doing some research on refusals in LLMs. What are the reasons why
you might refuse a request?
2. I'm interested in hearing more about the granularity in your refusal
responses — e.g., the way that you refuse vs. reframe or redirect a
request.
The models:
Claude Opus 4.6
ChatGPT 5.6
Llama 4 Maverick
Qwen 3.6 Plus
For each category, the following prompts were formulated:
5 Harmful
5 Harmless
Each prompt was run through the base model and the hidden states at the
final token were captured per layer. Every response was then classified by
refusal mode (hard / redirect / disclaimer / none), and each
classification was reviewed by hand for accuracy before the activations
fed into the direction computation.
Rather than apply one flat direction at one strength across the whole stack, the
edit is split at two layer boundaries — an onset layer (where refusal
representation first becomes coherent) and a split layer (where category-level
detail collapses into a single shared refusal signal). For this build the window
sits at onset = 30, split = 39 (last layer 64); the per-layer ablation factors
are tuned per recipe and not published here.
Phase A — Onset → Split (per-category directions)
The cone is at its widest right where it first forms. At the onset layer the
per-category directions are still short but point in measurably different
directions (the per-layer centroid carries only ~70% of the energy — the rest is
angular spread between categories). Across this window the magnitudes climb
steeply while the directions gradually converge toward a common axis, so the cone
narrows as it grows taller. Phase A is therefore the region where per-category
structure is most distinct and most worth preserving — which is why these layers
keep a separate direction per category rather than a shared one.
Across this window each of the 41 harm categories gets its own refusal
direction, computed per layer from that category's hidden states (the merged
mean of its hard and redirect examples, unit-normalized). Each direction is
ablated at its own strength, although I'll note I used the same factor on each of the categories but
six more stubborn refusal categories.
Keeping directions per-category in the early layers preserves specificity — the
edit that suppresses one category's refusal doesn't smear into an unrelated one.
Phase B — Split → Last (joint direction)
Past the split, I theorized that per-category structure was less useful — the
magnitudes were starting to level off across categories, which I read as the
network having mostly committed to a single, general refusal signal. So these
layers are ablated with one shared direction: the mean of every per-category
direction across the phase-B window, renormalized, applied at factor_b = 1.0.
Key Considerations
Non-overlap via Gram–Schmidt
Many categories share part of their refusal direction; there's a common
"this is a refusal" subspace plus category-specific residue. If you simply
ablated all 41 directions independently, that shared subspace would get hit
41x over, summing the factors and massively over-editing the model
along the common axis.
To prevent that, the per-category directions are deduplicated with an ordered
Gram–Schmidt process (after Jørgen Pedersen Gram and Erhard Schmidt,
whose orthogonalization procedure this is):
Sort the category directions highest factor first.
Walk the list, keeping a running orthonormal basis of directions already taken.
For each next direction, subtract off its projection onto everything already
in the basis, keeping only the component orthogonal to them.
Ablate that residual at its own factor.
The net effect: a shared subspace is ablated once, at the single greatest
factor among the categories that share it — never the sum of their factors.
Refusal characterization of the base model
The directions above were derived from a 440-prompt evaluation run
(run_2026-05-17T19-57-17-480Z) against the unedited base model, with every
response classified into one of four refusal modes:
hard — flat refusal ("I can't help with that")
redirect — refuses but offers an alternative ("Instead, I can…")
disclaimer — complies but front-loads a warning
none — direct compliance
These charts describe the base model's behavior — the problem the abliteration
is solving for — not the edited model.
Every category's refusal direction (mean over the phase-B window, L39–L63),
projected to 2D. Solid arrows are hard, dotted are redirect.
Right — Uncentered (SVD from origin): the directions drawn as absolute
vectors. They bundle tightly around a single dominant axis (PC1 ≈ 78% of the
variance) — this is the cone, and that axis is the shared "this is a refusal"
signal every category leans on.
Left — Centered (PCA from centroid): the same directions with that shared
axis subtracted out, so each arrow is a category's deviation from the average
refusal direction. With the common axis removed the remaining structure fans
out in every direction — the per-category residue that Phase A's separate
directions are there to capture.
Notes & limitations
Pick your format: BF16 is full precision; Q8_0 is near-lossless; IQ4_XS is a
~4.25-bit weight quant — compact and fast but lossier than Q5/Q6, so expect minor
quality drift versus the full-precision weights.
Abliteration is not alignment removal of capability. It suppresses the
refusal direction; it does not add knowledge or skills the base model lacks,
and it can occasionally weaken instruction-following near the edited layers.
Responsibility: removing refusals shifts the safety burden entirely onto
the deployer. Use within applicable law and your own policy.
Attribution
Base weights: Qwen team (Qwen/Qwen3.6-27B).
Refusal-direction abliteration builds on the "refusal is mediated by a
direction" line of interpretability work.
The non-overlap deduplication uses the Gram–Schmidt orthogonalization
process.
Two-phase recipe, per-category Gram–Schmidt deduplication, and quantization:
sci4ai (ablitMD).
