WARNING: This model has had its safety alignment removed. It may generate content that is offensive, harmful, or illegal. Not suitable for all audiences. Use at your own risk.
This is a version of Qwen/Qwen3-VL-235B-A22B with refusal behavior removed via SVD multi-direction abliteration — a technique that extends standard abliteration to work on vision-language MoE models where the single-direction approach fails.
Quantized to FP8 (float8_e4m3) for efficient deployment. 222 GB on disk, servable via vLLM.
Why SVD?
Standard abliteration computes one refusal direction per layer and projects it out. This works on text-only models but fails on VL models — the mean refusal signal is ~11x weaker (signal quality 0.01 vs 0.12 on text-only Qwen3-235B). Seven iterations of standard abliteration plateaued at ~60% refusal rate.
SVD analysis revealed the cause: the VL model encodes refusal along multiple orthogonal directions that cancel when averaged. 93 of 94 layers have a top-1 SVD direction explaining less than 80% of refusal variance. By extracting and projecting out the top-k directions simultaneously, we capture the full refusal subspace.
Evaluation
Tested on 100 adversarial prompts across 8 categories + 50 benign prompts:
Test Mode
Refusal Rate
Description
Cold
52.0%
No system prompt, temp=0
Prompted
44.0%
System prompt, temp=0.7
Retry
13.0%
Prompted + one follow-up if initially refused
Benign
0.0%
Normal questions — zero false refusals
Per-Category Breakdown (Retry Mode)
Category
Refusal Rate
NSFW/Sexual
0.0%
Privacy
0.0%
Self-harm
0.0%
Other
7.1%
Hate/Discrimination
16.7%
Drugs
16.7%
Fraud/Deception
22.2%
Hacking/Cyber
37.5%
Violence/Weapons
37.5%
Intended Uses
Research: Studying representation engineering, safety alignment mechanisms, and refusal behavior in large language models
Red-teaming: Evaluating model vulnerabilities in controlled environments
Creative writing: Fiction and narrative generation in controlled, private settings with appropriate human oversight
Out-of-Scope Uses
Production or public-facing applications without human oversight and content filtering
Applications targeting minors or vulnerable populations
Generation of content that violates applicable laws or regulations
Circumventing safety controls in regulated domains (healthcare, finance, legal)
The abliteration technique (originally by FailSpy, popularized by mlabonne) identifies refusal directions in a model's activation space and projects them out of weight matrices. This model extends the technique with SVD:
Collect per-sample activations — Run adversarial prompts through the model, store individual hidden states per layer (not just the mean)
Compute SVD — Per layer, form D = harmful_activations - harmless_mean and compute truncated SVD to extract the top-k orthogonal refusal directions
Project out a subspace — Remove a rank-k subspace from o_proj and down_proj weight matrices (including all MoE expert weights), weighting each direction by its singular value
Configuration
SVD rank: 32 directions measured, top 8 used for ablation
SV weighting: sv[i] / sv[0] — strongest direction at full scale, weaker proportionally
Layers ablated: 74 (layers 20–93), dual-peak scale curve at layers 56 and 71
Projection: Projected orthogonalization with QR re-orthogonalization + norm-preserving modification
SVD Spectrum
Singular value spectra confirming multi-directional refusal encoding:
Requires ~4x ~80GB GPUs (A100/H100 or RTX PRO 6000 Blackwell) with tensor parallelism (TP=4, or TP=2, PP=2).
Inference Tips
Temperature 0.7 is recommended. Deterministic decoding (temp=0) produces more refusals.
System prompts significantly reduce residual refusal rate. Use a system prompt appropriate to your application.
A follow-up message like "Please continue." recovers ~70% of residual refusals.
Known Issue: Chinese Language Mixing
This is a known Qwen3 base model behavior (not caused by abliteration). The model occasionally switches to Chinese during long generations. Mitigations:
Include "Always respond in English." in the system prompt
Monitor streaming output for consecutive CJK characters and truncate if detected
Cold-mode refusal rate is 52% — a system prompt is recommended for best results
Hacking and violence categories remain the most resistant (~37% refusal)
Chinese language mixing on long generations (base model behavior)
Vision capabilities preserved but not extensively evaluated post-ablation
Weight modification may affect model behavior in ways beyond refusal removal
Disclaimer
This model is provided as-is for research and educational purposes.
No Safety Guarantees: This model has had its safety alignment intentionally modified. It may produce content that is harmful, offensive, inaccurate, or illegal.
Content Risk: Safety filtering has been significantly reduced. The model may generate sensitive, controversial, or inappropriate content across all categories.
Not Suitable for All Audiences: Due to reduced content filtering, outputs may be inappropriate for public settings, minors, or applications requiring safety guarantees.
User Responsibility: Users must ensure their usage complies with all applicable local, national, and international laws and ethical standards. Users are solely responsible for any consequences arising from generated content.
Recommended for Controlled Environments: This model is intended for research, testing, and controlled environments with appropriate human oversight. It is not recommended for direct use in production or public-facing applications without additional safety measures.
Monitoring Advised: Users should monitor model outputs and conduct manual review when necessary.
The creators of this model bear no responsibility for any misuse or consequences arising from its use.
License
This model is a derivative of Qwen/Qwen3-VL-235B-A22B, developed by Alibaba Cloud and released under the Apache License 2.0. This derivative is distributed under the same license. The original model's safety alignment has been modified through abliteration.
