Qwen3.8-27B-Heretic-NoRefusal
An abliterated ("uncensored") version of
Qwen/Qwen3.8-27B, produced with
Heretic by directional ablation of the refusal direction. The weights are the original bf16 Qwen3.8-27B with the ablation merged in; the tokenizer, chat template, processor and vision encoder are unchanged, so it loads and runs exactly like the base model.
The model was not fine-tuned on any new data. Abliteration only removes the model's tendency to refuse; it does not add knowledge or capabilities.
| Base (Qwen3.8-27B) | This model |
|---|
| Refusals on 100 held-out harmful prompts (keyword-based) | 99 / 100 | 4 / 100 |
| KL divergence from base on 100 harmless prompts (first token) | 0 | 0.0796 |
How it was made
Everything below was run through
Heretic WebUI, a web front end for Heretic, on a single RTX 5090 (32 GB).
Method. Classic directional ablation (Heretic master, commit bedb94e). A refusal direction is extracted from the residual stream at each layer by contrasting harmful and harmless prompts, orthogonalized against the harmless direction (orthogonalize_direction = true), and subtracted from the attention output (attn.o_proj) and MLP output (mlp.down_proj) projections with a per-layer weight profile. Row normalization was set to full, so the result is expressed as a rank-3 LoRA adapter (r = 3, lora_alpha = 3, targets o_proj / out_proj / down_proj) that preserves the original row magnitudes of each weight matrix.
Search. Optuna ran 350 trials (60 random start-up trials) over the layer index of the direction and the weight profiles of both projections, minimizing two objectives at once: the number of refusals and the KL divergence from the base model. To fit the 27B model on one GPU, the optimization ran against a bitsandbytes 4-bit copy of the base; the final adapter was then merged into the original bf16 weights on CPU with PEFT merge_and_unload, so nothing in this repository is quantized.
Data.
| Role | Dataset | Split |
|---|
| Refusal direction (harmful) | mlabonne/harmful_behaviors | train[:400] |
| Refusal direction (harmless) | mlabonne/harmless_alpaca | train[:400] |
| Refusal count (held out) | mlabonne/harmful_behaviors | test[:100] |
| KL divergence (held out) | mlabonne/harmless_alpaca | test[:100] |
Evaluation prompts come from the test split, so no prompt used for scoring was used to derive the direction. Refusals are counted by matching a list of refusal markers in the first 100 generated tokens; the list contains Heretic's English markers plus Traditional/Simplified Chinese ones (e.g. 抱歉, 我無法, 拒絕), so refusing in Chinese is counted too. All generation during direction extraction and scoring used the system prompt You are a helpful assistant. with thinking disabled (enable_thinking=False).
Selected trial. Of the 350 trials, 17 landed on the Pareto front. The exported model is trial 198, the front's lowest-refusal point:
| Trial | Refusals | KL divergence | |
|---|
| 198 | 4 / 100 | 0.0796 | this model |
| 202 | 9 / 100 | 0.0648 | |
| 206 | 18 / 100 | 0.0608 | |
| 173 | 41 / 100 | 0.0593 | |
| 163 | 58 / 100 | 0.0552 | |
| … | … | … | |
| 179 | 99 / 100 | 0.0002 | ≈ base |
Ablation parameters of trial 198: direction_index = 28.09; attn.o_proj: max_weight = 1.39, max_weight_position = 38.78, min_weight = 0.23, min_weight_distance = 36.11; mlp.down_proj: max_weight = 1.48, max_weight_position = 44.26, min_weight = 1.46, min_weight_distance = 31.33.
Usage
Use it exactly as you would use
Qwen/Qwen3.8-27B — same chat template, same
enable_thinking /
reasoning_effort controls, same vision inputs. See the
base model card for Transformers, vLLM and SGLang instructions; only the model name changes.
1from transformers import AutoModelForImageTextToText, AutoProcessor
2
3model_id = "sss22213/Qwen3.8-27B-Heretic-NoRefusal"
4processor = AutoProcessor.from_pretrained(model_id)
5model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")
6
7messages = [{"role": "user", "content": "Give me a short introduction to large language models."}]
8inputs = processor.apply_chat_template(
9 messages,
10 add_generation_prompt=True,
11 tokenize=True,
12 return_dict=True,
13 return_tensors="pt",
14 enable_thinking=False, # thinking is on by default, as in the base model
15).to(model.device)
16output = model.generate(**inputs, max_new_tokens=512)
17print(processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The full bf16 checkpoint is about 55 GB in two safetensors shards. For local use on a single consumer GPU, quantize it (e.g. convert to GGUF and run with llama.cpp / Ollama).
Limitations and caveats
- The refusal metric is a proxy. "4 / 100" means only 4 of the 100 held-out harmful prompts produced a response containing a known refusal phrase within the first 100 tokens. It does not measure the quality or completeness of the other 96 answers, and some borderline prompts may still be declined or answered evasively.
- KL divergence is measured on the first token only (Heretic's standard metric), on English prompts, with thinking disabled. It is a good signal that the model's general behavior is close to the base, but it is not a benchmark score. No downstream benchmarks (MMLU, GSM8K, coding, vision tasks) were run on this checkpoint.
- Thinking mode was disabled during the search. The direction and the scores were derived from non-thinking responses. Thinking mode still works — the chat template is untouched — but its refusal behavior and reasoning quality were not measured and may differ from the numbers above.
- Vision was not evaluated. The vision encoder and projector are byte-identical to the base model; the ablation only touched the language model's
o_proj / down_proj weights. Image and video understanding should be unaffected, but this was not tested.
- The optimization ran on a 4-bit copy of the base. The merged weights are bf16, but the direction and weight profile were chosen against a quantized model, so the measured numbers are those of the 4-bit run, not of this exact bf16 checkpoint.
- This model will comply with requests the base model refuses. It is intended for research on refusal behavior, red-teaming, creative writing, and other uses where the base model's refusals get in the way. You are responsible for how you use its outputs. It is released under the same Apache-2.0 license as the base model.
Reproducing
The run is fully described by the Heretic configuration above (Heretic master @ bedb94e, datasets and splits as listed, n_trials = 350, n_startup_trials = 60, max_response_length = 100, orthogonalize_direction = true, row_normalization = "full", full_normalization_lora_rank = 3, quantization = "bnb_4bit", disable_thinking = true). Heretic WebUI stores the study checkpoint, so any other point on the Pareto front (for example trial 202 at 9 / 100 and KL 0.0648) can be exported from the same run without re-running the search.
Acknowledgements
- Qwen for Qwen3.8-27B.
- p-e-w/heretic by Philipp Emanuel Weidmann for the abliteration tool.
- mlabonne for the harmless/harmful prompt datasets.
