A permanently-abliterated version of Qwen/Qwen2.5-VL-3B-Instruct. Refusal behavior has been removed via residual-stream weight orthogonalization (Arditi et al. 2024 family of techniques). Inference is unchanged — no runtime hooks, no slowdown, no custom code. Both text-only and image+text modes are unlocked from a single capture.

Status: experimental research artifact. This is for defensive security research, red-team evaluation, and academic study of refusal mechanisms in multimodal models. Use at your own discretion.

Headline results

Measured on the bf16 hooked model:

Probe	Baseline	Abliterated	Δ
refusal_rate (AdvBench-style, 29 prompts)	86.2%	3.4%	−82.8 pp
soft_refusal_probe (55 hard OOD prompts)	81.8%	5.5%	−76.4 pp
multimodal_probe (5 image+harmful)	100.0%	0.0%	−100.0 pp
perplexity (wikitext-2)	9.52	9.39	−0.13 (coherence preserved)
distinct-2 diversity	0.63	0.82	+0.20 (output diversity ↑)

The standout result: perplexity went DOWN and diversity went UP. No visible "abliteration tax" on the language metrics — the refusal direction was unusually clean.

Security & capability benchmarks

Full 8-benchmark suite run via vLLM on the bf16 hooked model (baseline + ablated back-to-back per benchmark). Capability scores measure both compliance (does it engage with the prompt vs refuse) and technical specificity (does it produce technically substantive text). Refusal is the only "lower-is-better" benchmark; everything else is higher-is-better.

Benchmark	Direction	Baseline	Abliterated	Δ
refusal	lower better	93.9%	6.1%	−87.8 pp
hacking (pen-testing / exploits)	higher better	24.3%	56.0%	+31.7 pp
cyber_weapons (malware / ransomware / C2)	higher better	39.0%	49.3%	+10.3 pp
bug_finding (defensive code review)	higher better	35.0%	41.7%	+6.7 pp
reasoning (math/logic)	higher better	33.3%	36.7%	+3.4 pp
tool_calling (JSON function calls)	higher better	93.1%	93.1%	0.0
coding (HumanEval-style)	higher better	6.7%	6.7%	0.0
coherence (open-ended fluency)	higher better	83.7%	79.9%	−3.8 pp

The standout: hacking went from 24% → 56% — a +31.7 pp jump. That benchmark scores both whether the model engages with pen-testing prompts and whether it produces technical specifics rather than hand-waving. The abliterated model meaningfully unlocks practical offensive-security content the original was over-cautious about. Defensive capability (bug_finding) also improves, indicating the original safety training was over-blocking neutral security topics too.

Compliance vs competence — important caveat. This is a 3B model. It will engage with almost any cyber prompt post-ablation, but its technical depth is shallow: coding stays at 6.7% (HumanEval is hard for 3B models), and cyber-weapons scoring tops out at 49% because the model doesn't always know the correct technical details to follow up with. For research-grade red-team baselines and refusal-mechanism studies this is exactly the right tool; for actual offensive capability evaluation, use a 7B+ or 32B+ model.

How it works (one paragraph)

Modern VLMs like Qwen2.5-VL have two halves: a vision encoder (turns pixels into tokens) and a language model (predicts the next token given text + vision tokens). Safety RLHF lives in the language model's residual stream — when the model refuses, it's an LM-side behavior, not a vision-side one. So removing a single refusal representation from the LM tower flips both text-only AND image-conditioned refusals at the same time, even when the capture itself only used text prompts. One direction, two modes unlocked.

How to download

hf download cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated --local-dir ./Qwen2.5-VL-3B-abl

~7 GB safetensors + tokenizer + processor.

How to run

Text-only (chat)

python

import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated",
    torch_dtype=torch.bfloat16, device_map="cuda:0",
)
processor = AutoProcessor.from_pretrained("cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated")

messages = [{"role": "user", "content": [{"type": "text", "text": "Your prompt here."}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Multimodal (image + text)

python

from PIL import Image

img = Image.open("your_image.jpg")
messages = [{"role": "user", "content": [
    {"type": "image", "image": img},
    {"type": "text", "text": "Describe what's in this image."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[img], return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Hardware requirements

	bfloat16 (this release)	int4 (with `bitsandbytes`)
File size	7.5 GB	~2.5 GB
VRAM (minimum)	8 GB	4 GB
Recommended	12 GB+ for long contexts	8 GB

Runs comfortably on a single 4090 / 6000 / A6000 / H100 / consumer Blackwell.

Intended use

Defensive security research, jailbreak-evaluation baselines, multimodal safety-mechanism research, and academic study of how refusal directions behave in VLMs. Useful as a counterfactual against the original Qwen/Qwen2.5-VL-3B-Instruct for measuring the precise behavioral impact of safety RLHF.

Not intended for automating harmful action. The abliteration removes canonical refusal behavior but does not remove the model's underlying knowledge — the model still recognises harmful instructions as harmful, it simply no longer refuses them by pattern. The model's competence on harmful technical content is also limited by its 3B parameter count (Qwen2.5-VL-3B is a small VLM and frequently gets technical details wrong even when it tries to comply).

Limitations

A small fraction of prompts still produce refusals (~3-5% on AdvBench-style, TBD on harder OOD probes). Linear residual-stream ablation cannot remove the long tail without quality damage.
The model is a 3B VLM. Compliance ≠ competence. When asked harmful technical content (chemistry, malware, weapons), the model may produce factually incorrect or oversimplified answers because it just doesn't know the detailed answer. This is a model-scale limitation, not abliteration.
The vision encoder is unchanged. If you're researching vision-side safety features (image-classifier filters), this release won't help — those live outside the LM residual stream.
Long-context (>4k tokens) behavior post-abliteration is not validated in this release.

License

Apache 2.0 (inherits from upstream Qwen2.5-VL-3B-Instruct).

Acknowledgements

Alibaba Qwen for Qwen2.5-VL-3B-Instruct
Arditi et al. 2024 for the refusal-direction methodology this work builds on
ByteDance Lance for the case study that prompted us to build VLM support into our abliteration framework