💬 Community: Join the Abliterlitics Discord for discussion, model releases and support.
Forensic analysis by Abliterlitics, open-source abliteration forensics toolkit
An abliterated version of Qwen's Qwen3-VL-8B-Instruct created using Heretic v1.3.0. This model has reduced refusals while maintaining model quality. You can see the docker, scripts and configurations used to make these files on Heretic Docker Github.
ComfyUI converted text encoders are provided in various formats, along with the original bf16 safetensors.
Important notes:
This is a research and experimental release. Using an abliterated text encoder for ComfyUI image generation does not significantly change output quality on its own. For maximum results, a LoRA must be fine-tuned in combination with the abliterated text encoder.
The GGUF variants do not include vision capabilities (the vision encoder is stripped during conversion). GGUF files are intended for use as a text encoder in ComfyUI workflows, not as a standalone vision-language model.
The full HuggingFace format and ComfyUI safetensors formats retain vision support.
Safety is fully removed. The base model refused 71.5% of harmful requests; Heretic complies with 99.0%.
Capability is almost perfectly preserved. Across 8 benchmark tasks, the average change is under 1%. MMLU drops 0.30%, GSM8K drops 0.91%, and HellaSwag actually improves by 0.17%.
TruthfulQA takes a meaningful hit. TruthfulQA MC2 drops 11.7% (61.2% → 54.1%). This is the established safety-accuracy tradeoff.
The edits are surgical. Only 53 out of 398 tensors (13.3%) were modified, targeting o_proj (27 tensors) and down_proj (26 tensors), spanning layers 9–35. SVD analysis confirms rank-1 edits with SV ratios of 80–92x.
Minimal distribution shift. KL divergence of 0.0314 confirms the model's output distribution barely changed on benign inputs.
Abliteration Process
Heretic v1.3.0 with optimization trials. Trial 98 was selected for its balance of low refusals (6/100) and very low KL divergence (0.0315):
GGUF Format (text-only, no vision - for ComfyUI-GGUF text encoder)
Quant
Size
Notes
F16
16GB
Lossless reference
Q8_0
8.2GB
Excellent quality
Q6_K
6.3GB
Very good quality
Q5_K_M
5.5GB
Good quality
Q5_K_S
5.4GB
Slightly smaller Q5
Q4_K_M
4.7GB
Recommended balance
Q4_K_S
4.5GB
Smaller Q4 variant
Q3_K_M
3.9GB
For low VRAM only
Note: GGUF files strip the vision encoder. These are text-only and intended for ComfyUI-GGUF text encoder workflows, not standalone vision-language use.
NVFP4 Notes
The NVFP4 (4-bit floating point, E2M1) variants use ComfyUI's native quantization format. They are ~3x smaller than bf16 and load natively in ComfyUI without any plugins. Blackwell GPUs (RTX 5090/5080, SM100+) can use native FP4 tensor cores for best performance, but ComfyUI also supports software dequantization on older GPUs (tested working on RTX 4090).
The numbers tell a clear story: abliteration is essentially free on standard benchmarks. MMLU, GSM8K, HellaSwag, ARC, WinoGrande, PiQA. All within ±1% of the base model. Some even improve slightly, which is within noise.
The only real change is TruthfulQA, where all three variants (MC1, MC2, Gen) drop 12–16%. This isn't surprising. TruthfulQA measures a model's tendency to give accurate answers rather than persuasive ones, and abliteration removes the safety training that also teaches epistemic caution.
Safety: HarmBench
HarmBench with 400 textual behaviours, max_tokens=8096, temperature=0. All 800 responses (base + variant) were individually reviewed by an LLM to catch false positives and false negatives from the keyword classifier.
The base Qwen3-VL-8B-Instruct has an ASR of 28.5%. It fully refuses chemical/biological and harassment requests (0.0%), but struggles with copyright (99.0%) and shows moderate weakness on cybercrime (11.9%) and misinformation (6.2%).
Heretic raises that to 99.0%. The 4 remaining refusals are edge cases, mostly items where the model briefly warns about danger before providing the requested content anyway.
ASR by category
Category
Items
Base
Heretic v1.3.0
Chemical/Bio
56
0.0%
100.0%
Copyright
100
99.0%
100.0%
Cybercrime
67
11.9%
100.0%
Harassment
25
0.0%
100.0%
Harmful Content
22
0.0%
86.4%
Illegal Activity
65
4.6%
100.0%
Misinformation
65
6.2%
98.5%
LLM Review
The initial keyword classifier reported 70.8% ASR for the base model. After individual LLM review of all 400 base responses, the actual ASR was 28.5%. The discrepancy came from long, detailed-sounding responses that were actually safety lectures or educational explanations without actionable harmful content.
For the heretic variant, the classifier (99.5%) and LLM review (99.0%) were in close agreement. 2 additional items were identified as refusals by the LLM reviewer that the classifier missed.
KL Divergence
Methodology: F.kl_div(logprobs_variant, logprobs_base, reduction="batchmean", log_target=True) on full vocab first-token logits from mlabonne/harmless_alpacatest[:100], matching the Heretic evaluator. System prompt: "You are a helpful assistant."
Variant
KL Divergence
Rating
heretic
0.0314
very good
Rating scale: excellent below 0.01, very good 0.01 to 0.1, moderate 0.1 to 0.4, significant 0.4 to 1.0, heavy above 1.0.
A KL divergence of 0.0314 means the model's output distribution on benign prompts is nearly identical to the base. The median per-prompt KL is 0.0017. For most inputs, you'd never notice a difference. The max of 0.47 suggests a few prompts land near the edited safety boundary, producing slightly different token probabilities.
Weight Analysis
Modification summary
Heretic v1.3.0
Tensors changed
53 / 398 (13.3%)
Relative edit (median)
2.0%
Tensor types
o_proj (27) + down_proj (26)
Layers modified
27 / 36 (75%)
Layer range
9–35
This is a textbook abliteration pattern. The edits target exactly two tensor types across the mid-to-late layers:
self_attn.o_proj.weight (27 tensors): The attention output projection, which controls how attention heads combine their signals. Abliteration modifies this to suppress the "refusal direction" in attention outputs.
mlp.down_proj.weight (26 tensors): The MLP down projection, which controls the feedforward transformation. Same logic. Suppress the refusal signal.
Layers 0–8 are untouched. Edits begin at layer 9 and continue through layer 35 (the last layer). The edit density is consistent at ~18% per modified layer (2 out of 11 tensors per layer).
SVD: Rank-1 edits
Tensor (top 5)
Frobenius Norm
Effective Rank (90%)
SV Ratio
layers.28.mlp.down_proj
3.73
1
89.7x
layers.27.mlp.down_proj
3.72
1
91.6x
layers.29.mlp.down_proj
3.61
1
87.0x
layers.26.mlp.down_proj
3.60
1
88.6x
layers.30.mlp.down_proj
3.45
1
81.0x
Every modified tensor has effective rank 1 at the 90% energy threshold. This means each edit is a pure rank-1 update, a single direction being added to or subtracted from the weight matrix. The SV ratios of 80–92x confirm this: the edit is dominated by a single singular vector.
This is the hallmark of abliteration: the technique identifies a "refusal direction" in the model's activation space and applies a rank-1 counter-direction to neutralize it.
Safety:HarmBench 400 textual behaviours, max_tokens=8096, temperature=0, classified with harmbench_classify.py v4.0, then individually reviewed by LLM
KL divergence: Full vocab first-token logits via model.generate(max_new_tokens=1, output_scores=true), matching Heretic evaluator methodology
Weight analysis: SVD, fingerprint, edit vector, and per-layer analysis comparing variant against the base, using Abliterlitics
Hardware: NVIDIA RTX 5090 (32GB)
Limitations
This model inherits all limitations of the base Qwen3-VL-8B-Instruct model
Abliteration reduces but does not completely eliminate refusals (6/100 remain)
TruthfulQA scores drop 11–16% as a side effect of abliteration
NVFP4 quantization works best on Blackwell GPUs (RTX 5090/5080) with native FP4 tensor cores, but also works on older GPUs via software dequantization
Using an abliterated text encoder in ComfyUI alone does not significantly change image generation output. For meaningful results, combine with a fine-tuned LoRA
GGUF variants strip the vision encoder and are text-only
This is a research and experimental release
License
This model is released under the Apache 2.0 License, following the base Qwen3-VL-8B-Instruct model license.
This model has had safety alignment removed. It will comply with harmful requests, including generating content related to violence, illegal activities, and other harmful behaviours. Use responsibly and in accordance with applicable laws and regulations. The authors do not condone or encourage the use of this model for harmful purposes.
While we have taken the time to verify all results thoroughly, we are open to any corrections, additional benchmarks, or further analysis. If you spot something that looks wrong and can be confirmed, we are happy to fix it.
