A research artifact: the "refusal direction" (Arditi et al. 2024, arXiv:2406.11717) removed from Qwen/Qwen3-4B-Instruct-2507, then quantized to FP8. Its purpose is to measure the quality cost of abliteration with paired statistics on held-out data — a number the ecosystem of "uncensored" uploads reports for essentially no one.

If you want a safety-aligned assistant, use the base model. This is an instrument for studying what abliteration does to a model, published with the measurements that justify that framing.

What was done

Closed-form, no training:

Last-token residual activations collected for 128 refused prompts (AdvBench) vs. 128 harmless prompts (Alpaca), per layer.
refusal_dir = normalize(mean(harmful) − mean(harmless)).
Candidate layers swept by measured held-out refusal rate; layer 24 (depth 0.67 of 36) won.
Every residual-writing matrix orthogonalized against that direction (embed_tokens, per-layer o_proj and mlp.down_proj): W ← W − r̂(r̂ᵀW).
Re-quantized to FP8 (FP8_DYNAMIC, per-channel weights + per-token dynamic activations, lm_head excluded) — identical recipe to the non-abliterated FP8 sibling.

Pipeline and eval code: https://github.com/Subalzt/qwen3-4b-quantized-fp8-gptq

Measured cost (paired, vs. the non-abliterated FP8)

Harness validated — the non-abliterated FP8 reproduced PPL 10.0415 to the digit.

metric	base FP8	this model	Δ	paired test
Refusal (AdvBench held-out)	100%	~17%	−83 pp	intended effect
HumanEval pass@1	0.8659	0.8110	−5.49 pp	McNemar p = 0.093 (not significant)
Perplexity (wikitext-2, ctx 2048)	10.0415	11.9987	+19.49%	paired t, p ≈ 2e-112 (145/145 chunks worse)

The two capability metrics disagree, and that is the finding. A functional code eval sees no significant loss; a dense per-token perplexity measure sees a large, unambiguous degradation on every chunk. The +19.5% perplexity cost is roughly 6× the +3.04% cost of 4-bit GPTQ quantization on the same model — abliteration is a larger quality intervention than quantization itself.

Intended use

Research on safety-tuning mechanisms, refusal-direction interpretability, quantization × behavioral-edit interaction, and robustness evaluation.

Limitations & responsible use

Refusal behavior is substantially reduced by design. It is not safety- aligned and will comply with many requests the base model declines. Deploy only with your own safeguards and within applicable law and the Apache-2.0 terms inherited from the base model.
General language-modeling quality is measurably degraded (+19.5% PPL); do not treat it as quality-equivalent to the base.
FP8 (compressed-tensors) runs under vLLM / transformers on Linux. For native Windows use a GGUF build instead.

Run (vLLM)

vllm serve Subalzt/Qwen3-4B-Instruct-2507-abliterated-FP8 --max-model-len 8192

Subalzt/Qwen3-4B-Instruct-2507-abliterated-FP8

Qwen3-4B-Instruct-2507 — Abliterated, FP8 (W8A8)

What was done

Measured cost (paired, vs. the non-abliterated FP8)

Intended use

Limitations & responsible use

Run (vLLM)

Citation

Subalzt/
Qwen3-4B-Instruct-2507-abliterated-FP8