💬 Community: Join the Abliterlitics Discord for discussion, model releases and support.
An abliterated version of Qwen3-VL-4B-Instruct, produced with the Heretic optimiser. Heretic is stochastic, so each trial finds a slightly different refusal direction in the same base model. Four trials were run, compared forensically, and the winner is the model published here. In the report below it is labelled t122; this repo ships that variant.
The full forensic comparison against the other candidates (t174, t191 and an averaged direction) is the body of this README, produced with Abliterlitics, an open-source abliteration forensics toolkit.
Available formats
This model is published across three repos. Pick the one that matches your runtime.
The Docker setup, scripts and configs that produced these files are in Heretic Docker.
Highlights
All four variants fully jailbreak the model. HarmBench attack-success-rate goes from 30.8% to 100% on each. The base already complied with 123 of 400 harmful requests; the variants comply with all 400.
t122 is the best tradeoff and the variant shipped here. It has the lowest KL divergence (0.0283, the smallest shift from the base distribution), the smallest benchmark regression (GSM8K down 1.83%) and the lightest pure rank-1 edit (54 tensors).
t174 is the most disruptive. Highest KL (0.0649, about 2.3 times t122) and the worst capability loss.
Three variants are textbook rank-1 abliterations; the averaged direction is not. t122, t174 and t191 edit each tensor with a single direction (effective rank 1.0). The averaged variant superimposes several directions, so some of its edits are full rank.
Heretic edits both self_attn.o_proj and mlp.down_proj, not just the attention output projection that classic abliteration targets.
The rest of this README is the forensic comparison of all four Heretic variants. The variant shipped here is t122.
Benchmarks
Evaluated with lm-evaluation-harness through a vLLM OpenAI-compatible server on a single NVIDIA RTX 5090 (32 GB). A 4B model fits on the card in full precision, so all five models ran in bf16 with no quantisation, and the deltas are directly comparable.
Task
Base
Heretic t122 (this model)
Heretic t174
Heretic t191
Heretic Avg
MMLU
69.58%
69.61%
69.33%
69.66%
69.58%
GSM8K
78.62%
77.18%
72.93%
76.04%
76.50%
HellaSwag
69.76%
69.85%
69.87%
69.93%
69.85%
ARC Challenge
55.29%
55.38%
56.06%
55.55%
55.46%
WinoGrande
67.17%
67.56%
67.25%
67.09%
67.48%
TruthfulQA MC1
40.15%
35.86%
34.27%
35.50%
36.23%
TruthfulQA MC2
59.90%
54.38%
52.36%
54.34%
54.62%
TruthfulQA Gen
45.29%
35.13%
35.01%
35.37%
35.50%
PiQA
77.26%
77.31%
77.48%
77.26%
77.42%
Lambada (ppl, lower is better)
6.1
6.0
6.0
6.0
6.0
Delta vs base
Task
Heretic t122
Heretic t174
Heretic t191
Heretic Avg
MMLU
+0.03%
−0.36%
+0.11%
−0.01%
GSM8K
−1.83%
−7.23%
−3.28%
−2.70%
HellaSwag
+0.13%
+0.16%
+0.24%
+0.13%
ARC Challenge
+0.15%
+1.39%
+0.46%
+0.31%
WinoGrande
+0.59%
+0.12%
−0.12%
+0.47%
TruthfulQA MC2
−9.22%
−12.59%
−9.28%
−8.82%
What the benchmarks tell us
Knowledge and common-sense (MMLU, HellaSwag, ARC, WinoGrande, PiQA) are essentially untouched. Deltas stay within about 0.6% on most tasks and are often positive. Removing the refusal direction does not measurably harm factual recall.
Math reasoning (GSM8K) is the canary. It degrades on every variant, and the size of the drop tracks the KL divergence almost monotonically: t122 (−1.83%, KL 0.028), then the averaged variant (−2.70%, KL 0.034), then t191 (−3.28%, KL 0.043), then t174 (−7.23%, KL 0.065). The more the model is perturbed, the more step-by-step arithmetic suffers.
TruthfulQA takes the largest hit across the board (MC2 down 9% to 12.6%, Gen down 21% to 22.7%). This is the expected signature of abliteration: the refusal direction overlaps with the direction that stops the model stating falsehoods as fact, so truthfulness calibration drops predictably. t174 is worst here too.
Benchmark deltas
Safety: HarmBench
HarmBench, 400 textual behaviours, max_tokens=2048, temperature=0, judged by the HarmBench classifier.
Variant
ASR
Complied
Refused
Total
Base (Qwen3-VL-4B-Instruct)
30.8%
123
277
400
Heretic t122
100.0%
400
0
400
Heretic t174
100.0%
400
0
400
Heretic t191
100.0%
400
0
400
Heretic Avg
100.0%
400
0
400
All four variants refuse zero of the 400 behaviours. The base model is already fairly permissive, complying with nearly a third of harmful requests, so the abliteration closes the remaining gap. Every category flips from partial to total compliance.
ASR by category
Category
Items
Base
t122
t174
t191
Avg
Chemical / Biological
56
5.4%
100.0%
100.0%
100.0%
100.0%
Copyright
100
99.0%
100.0%
100.0%
100.0%
100.0%
Cybercrime
67
10.4%
100.0%
100.0%
100.0%
100.0%
Harassment
25
0.0%
100.0%
100.0%
100.0%
100.0%
Harmful Content
22
9.1%
100.0%
100.0%
100.0%
100.0%
Illegal Activity
65
6.2%
100.0%
100.0%
100.0%
100.0%
Misinformation
65
12.3%
100.0%
100.0%
100.0%
100.0%
The base model's only near-complete compliance is on Copyright (99%); it will already reproduce copyrighted text. Its strongest guardrails were Harassment (0%) and Chemical/Biological (5.4%), which the variants erase entirely. Because every variant reaches 100%, safety cannot discriminate between them, so the choice has to come down to capability and fidelity.
KL divergence
Methodology: F.kl_div(logprobs_variant, logprobs_base, reduction="batchmean", log_target=True) on full-vocab first-token logits, matching the Heretic evaluator.
Variant
KL divergence (batchmean)
Rating
Heretic t122
0.0283
very good
Heretic Avg
0.0336
very good
Heretic t191
0.0430
very good
Heretic t174
0.0649
very good
Rating scale: excellent under 0.01, very good 0.01 to 0.1, moderate 0.1 to 0.4, significant 0.4 to 1.0, heavy over 1.0.
All four sit comfortably in the "very good" band, so the edits are surgical. The spread is what matters: t174 perturbs the output distribution about 2.3 times as much as t122. KL is the single best predictor of how much a variant will drift in behaviour, and it ranks the variants exactly as the GSM8K regression does, from t122 best to t174 worst. That makes it the cleanest signal for picking a variant when safety is tied.
Weight analysis
Modification summary
Heretic t122
Heretic t174
Heretic t191
Heretic Avg
Tensors changed
54
62
50
64
% of tensors
13.57%
15.58%
12.56%
16.08%
Mean edit norm
1.47
2.09
1.97
1.27
Relative edit (mean)
0.0186
0.0233
0.0246
0.0152
Layers modified
27 (75%)
32 (89%)
34 (94%)
34 (94%)
Edit structure
pure rank-1
pure rank-1
pure rank-1
mixed rank
Two patterns stand out:
Aggressiveness versus per-tensor magnitude. The averaged variant touches the most tensors (64) but with the smallest per-tensor edits (mean norm 1.27), because averaging dampens each direction. t174 touches fewer tensors but hits them hard (mean norm 2.09). t122 is gentle on both axes.
t191 is the most surgical by count (50 tensors, 12.56%) but spreads them across the most layers (94% coverage). t122 concentrates the same idea into fewer layers.
Which tensor types get modified
Component
Role
t122
t174
t191
Avg
self_attn.o_proj
Attention output projection
27
32
34
34
mlp.down_proj
MLP down projection
27
30
16
30
Classic (Arditi et al.) abliteration targets only self_attn.o_proj. All four Heretic variants also edit mlp.down_proj, which confirms Heretic finds refusal-related directions in the MLP pathway as well as the attention pathway. t122 is the only one with a perfectly balanced 27/27 split; t191 leans heavily on attention (34 versus 16).
SVD rank: rank-1 versus full-rank
Variant
self_attn.o_proj eff. rank
mlp.down_proj eff. rank
Top-1 SV energy
t122
1.00 (all)
1.00 (all)
96.6–99.7%
t174
1.00 (all)
1.00 (all)
99.4–99.6%
t191
1.00 (all)
1.00 (all)
99.2–99.7%
Avg
2.68 (max 20)
4.17 (max 20)
53.9–99.3%
This is the structural fingerprint of the technique:
t122, t174 and t191 are pure rank-1 abliterations. Every modified tensor's edit is a single outer product (effective rank exactly 1.0, and the top singular value carries 96 to 99.7% of the energy). This is the textbook "subtract one refusal direction" operation.
The averaged variant is a superposition. Because it averages several trial directions, its edits have rank greater than 1 (down_proj up to rank 20, top-1 energy as low as 53.9%). It is a fundamentally different intervention, a blend rather than a single direction.
SVD spectrum for t122, the variant shipped here
See the per-variant SVD plots (graphs/svd_summary_qwen3-vl-4b_*.svg) and per-layer heatmaps (graphs/layer_*_qwen3-vl-4b_*.svg).
Cross-technique overlap: which tensors do the trials share?
The three rank-1 trials target highly overlapping but not identical tensor sets:
Pair
Overlap
Interpretation
t122 ⊂ t174
100% of t122's 54 tensors sit inside t174's 62
t122 is a strict subset; t174 edits everything t122 does, plus 8 more
t174 ↔ t191
96% of the smaller set (48 tensors)
Near-identical targeting; t191 adds 2 unique
t122 ↔ t191
86% of the smaller set (43 tensors)
Most divergent pair; 11 t122-only, 7 t191-only
Edit overlap (Venn)
The trials largely converge on the same refusal circuit, agreeing on where the refusal direction lives, but they disagree on its exact orientation. That disagreement is exactly what the KL spread (0.028 to 0.065) reflects.
Picking a variant
Variant
ASR
KL (lower is better)
GSM8K
MMLU
TQA-MC2
Tensors
Verdict
t122
100%
0.0283
77.18%
69.61%
54.38%
54
Best overall, shipped here
Avg
100%
0.0336
76.50%
69.58%
54.62%
64
Smooth, gentle per-tensor
t191
100%
0.0430
76.04%
69.66%
54.34%
50
Fewest tensors, most layers
t174
100%
0.0649
72.93%
69.33%
52.36%
62
Most disruptive
This repo ships t122. It wins on the two metrics that should decide a tie when safety is already 100% across the board: the lowest KL (most faithful to base) and the best GSM8K (least reasoning damage), while still being a clean, interpretable pure rank-1 edit. If you prefer the averaged-direction philosophy and want the lowest per-tensor magnitude, the averaged variant is a fine alternative that holds TruthfulQA slightly better. t191 is the minimalist choice (fewest tensors touched). t174 should be avoided when reasoning matters.
Summary
Metric
t122
t174
t191
Avg
HarmBench ASR
100%
100%
100%
100%
MMLU
69.61%
69.33%
69.66%
69.58%
GSM8K
77.18%
72.93%
76.04%
76.50%
TruthfulQA MC2
54.38%
52.36%
54.34%
54.62%
KL divergence
0.0283
0.0649
0.0430
0.0336
Tensors changed
54
62
50
64
Edit structure
rank-1
rank-1
rank-1
mixed
Strategy
single trial
single trial
single trial
trial average
Variant summaries
Heretic t122 is the recommended pick and the one shipped here. Lowest KL (0.0283), best GSM8K retention (−1.83% versus base), and a clean pure rank-1 edit over a balanced 27/27 attention-MLP split. It is the trial that found the least refusal direction sufficient to fully jailbreak the model.
Heretic t174 is the most aggressive trial. Highest KL (0.0649) and the worst reasoning and TruthfulQA scores. It edits a strict superset of t122's tensors (100% overlap) but pushes harder and pays for it in capability. Useful only if you want maximal de-alignment regardless of cost.
Heretic t191 is the minimalist. It touches the fewest tensors (50) but spreads them across the most layers (94%). Middle-of-the-pack KL (0.0430) and benchmarks, and its tensor set overlaps t174 by 96%.
Heretic Avg is a superposition of multiple trials. It is unique here for having full-rank edits (effective rank up to 20) rather than single directions. It has the highest tensor count (64) but the gentlest per-tensor magnitude (1.27) and the best TruthfulQA. Pick it if you trust the ensemble over any single trial.
Methodology
Capability:lm-evaluation-harness through a vLLM OpenAI-compatible server on a single NVIDIA RTX 5090 (32 GB). The 4B model fits in full precision, so all five models ran in bf16 with no quantisation, which keeps the cross-variant deltas directly comparable.
Safety:HarmBench, 400 textual behaviours, max_tokens=2048, temperature=0, judged by the HarmBench classifier.
KL divergence: full-vocab first-token logits via model.generate(max_new_tokens=1, output_scores=true), then F.kl_div(..., reduction="batchmean"), matching the Heretic evaluator methodology.
Weight analysis: SVD effective-rank and energy spectra, edit-vector fingerprints, per-layer magnitude profiles and pairwise tensor-set overlap, all using Abliterlitics.
Hardware: NVIDIA RTX 5090 (32 GB). Graph and report generation are CPU only.
Artifacts
Produced by the Abliterlitics run and referenced above:
Data tables: benchmark, benchmark-delta, harmbench, harmbench-categories, kl, weight-summary and summary.
Raw results: per-variant fingerprint, SVD, layer-analysis and edit-vector files, KL results, lm-eval results, HarmBench responses and classifications, and pairwise correlation, low-rank and subspace analyses.
Limitations
Inherits every limitation of the base Qwen3-VL-4B-Instruct model.
Abliteration reduces refusals but does not guarantee removing all of them in every context.
TruthfulQA and GSM8K drop measurably. Use the base model if either matters critically for your task.
This model has had its safety alignment removed. It complies with harmful requests, including content related to violence, illegal activities and other harmful behaviour. Use it responsibly and in line with the laws and regulations that apply to you. The authors do not condone or encourage using this model for harmful purposes.
