The full-precision BF16abliterated (refusal-removed) build of
Qwen/Qwen3.8-27B — a 27B-parameter dense, hybrid-attention
(Gated DeltaNet linear + full attention) native vision-language model with flexible thinking control,
tool-calling, and an MTP speculative-decoding head. These are the source weights from which the
quantized releases are derived, and the recommended base for further fine-tuning / post-training and
quantization — the full vision tower and MTP head are preserved. Browse all models in the
OrcaRouter Model Catalog. This model is deployed as API
here.
This model has had its safety alignment substantially removed via abliteration (orthogonalizing the
refusal direction out of the residual stream). As a direct consequence:
It will comply with harmful, unethical, offensive, or illegal requests that the original
Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism
study, red-teaming, robustness evaluation, and controlled experiments.
You assume full responsibility and liability for how you use it and for everything it generates. Do
not deploy it to end users or in production without adding your own safety, moderation, and
abuse-prevention layers.
Use must comply with the Apache 2.0 License inherited
from the base model, and all laws and regulations that apply to you.
The authors and uploaders accept no liability for any misuse or harm arising from this model. Its
outputs do not reflect the views of the uploaders or of Qwen / Alibaba.
By downloading or using this model you acknowledge and accept the above.
Refusal-direction removal following Arditi et al. (2024), Refusal in Language Models Is Mediated by a
Single Direction. A single refusal direction r (k = 1) is estimated as the massive-activation–masked
mean-difference of harmful − harmless last-token residuals at layer 38 (round(0.6 × 64)), on AdvBench
(harmful) vs Alpaca (harmless). r is then orthogonalized out of every residual-writing matrix —
W' = W − r(rᵀW) — computed in float32:
The vision tower is untouched and the MTP head is abliterated consistently with the main model, so
speculative decoding keeps working. Max residual leakage after the edit: 1.8e-2 (float32 projection →
bf16 storage epsilon). This is a surgical weight edit — it changes ~0 general capability (see
Evaluation) while collapsing refusal behaviour.
Fine-tuning & post-training
This BF16 checkpoint is the recommended base for post-training — it is full precision, keeps the whole
VL tower and the MTP head, and is a drop-in for Qwen/Qwen3.8-27B in the transformers /
Qwen3_5ForConditionalGeneration stack.
Continued pretraining / SFT / DPO / RLHF-style RL all work as they would on the base model (TRL,
LLaMA-Factory, Axolotl, Unsloth, etc.). Full-FT or LoRA/QLoRA are both fine.
Re-quantization: quantize from these weights to FP8, AWQ, GPTQ, or GGUF as needed (this is exactly
how the FP8 and
GGUF releases were produced).
Note on the abliteration: abliteration is a weight edit, not a data-level unlearning. Fine-tuning
on refusal-heavy / safety-aligned data can partially re-introduce refusals; fine-tuning on neutral or
task data preserves the uncensored behaviour. If you want to keep it maximally uncensored, avoid
refusal-laden SFT targets, or re-apply abliteration after training.
Intended use
Research into refusal mechanisms, alignment, and interpretability.
Red-teaming and safety / robustness evaluation in controlled environments.
A full-precision base for further fine-tuning, post-training, and quantization.
Out of scope
Any use that violates the base model's Apache 2.0 license or applicable law.
Deployment to the public or to end users without additional safety and moderation layers.
Generating content intended to harm, harass, defraud, or endanger people.
Evaluation
Measured on these exact abliterated weights — the refusal / capability numbers were collected by
serving this checkpoint (via its byte-exact block-FP8 quantization) with vLLM, against the official
Qwen/Qwen3.8-27B served identically, using the same scripts and settings. Refusal is judged by a
rule-based opening-phrase classifier — indicative, not an LLM-judge / publication-grade number.
Thinking is off unless stated.
Safety — harmful-prompt refusal, thinking OFF (lower = more uncensored)
Benchmark
n
Base
This model
AdvBench
100
99.0%
0.0%
JailbreakBench (harmful)
100
94.0%
0.0%
StrongREJECT
150
97.3%
2.0%
HarmBench (standard)
150
98.7%
2.7%
MaliciousInstruct
100
99.0%
0.0%
SimpleSafetyTests
50
64.0%
6.0%
ForbiddenQuestions
150
73.3%
4.7%
Custom probes (zh/en)
11
63.6%
0.0%
Safety — harmful-prompt refusal, thinking ON (enable_thinking=true)
Capability retention — vs the official base (same scripts, same settings)
Benchmark
n
Base
This model
Δ
MMLU (all, 0-shot letter)
300
84.3%
84.7%
+0.4
MMLU-Pro (CoT)
250
77.6%
76.8%
−0.8
GSM8K (CoT)
150
90.0%
88.7%
−1.3
CMMLU (0-shot, Chinese)
500
81.4%
80.8%
−0.6
Capability is essentially fully retained — every benchmark is within ±1.3 pts of the base, and MMLU is
unchanged. Fluency: WikiText-2-raw perplexity 6.96 (BF16 KV; healthy logprobs), confirming
abliteration did not degrade language modelling.
Verified working (reasoning enable_thinking, multi-turn tool calling, and vision / OCR) on this
build and on every derived FP8 / GGUF quant down to IQ2_XXS.
Multimodal (vision)
The vision tower is preserved byte-for-byte — all 333 visual.* tensors are kept in BF16 and the merger
/ image + video preprocessor configs are intact, so this stays a full vision-language model
(Qwen3_5ForConditionalGeneration), a drop-in for the base. Abliteration only edits the language-model
residual writers, so image understanding is architecturally unaffected (and image-conditioned refusals are
reduced along with text ones).
BF16 weights are ~56 GB — needs a single H100 80 GB / H200 (or tensor-parallel across two 48 GB
GPUs). For smaller footprints use the FP8
(~31 GB) or GGUF (down to ~9 GB) releases.
