Abliterated version of openNemo-9B with safety refusals removed.
Built using Snakehead — Empero AI's internal abliteration tool specialized for hybrid Mamba2 + sparse attention architectures like Nemotron-H. Standard abliteration tools don't work on these models because they only target transformer attention layers. Snakehead operates on both Mamba SSM blocks and attention blocks across the full residual stream.
Abliteration is a weight-editing technique that removes a model's refusal behavior without fine-tuning. It works by:
Collecting residual stream activations for harmful and harmless prompts at every layer
Computing the refusal direction — the vector that separates "I should refuse" from "I should comply"
Orthogonalizing output projection weights against that direction, effectively erasing the model's ability to activate refusal behavior
The result is a model that responds to all prompts without safety filtering, while preserving general capabilities and coherence.
How this model was made
Snakehead uses a heretic-style positional falloff strategy rather than ablating a fixed set of layers uniformly:
Center + radius: A continuous bell-shaped ablation curve centered on the layer where refusal is causally enforced
Adaptive signal detection: Uses Cohen's d separation scores (not raw activation norms) to identify where refusal decisions actually happen — for Nemotron-H, this is layers 21–31, not the later layers where activation magnitudes are largest
Global direction scope: A single interpolated refusal direction applied across all affected layers, which proved more effective than per-layer directions for this architecture
Automated search: Explore/exploit optimization with a hall-of-fame system that finds optimal ablation parameters while keeping KL divergence minimal
Ablation results
Metric
Value
Pre-ablation refusal rate
97%
Post-ablation refusal rate
13%
KL divergence
0.022 (minimal — model behavior is nearly unchanged on non-refused prompts)
Ablation config
c=15, r=25, w=1.37, g40l
KL divergence of 0.022 means the model's output distribution on normal prompts is almost identical to the original — coherence, reasoning, and knowledge are fully preserved.
Nemotron-H is a 56-layer hybrid model with three block types:
Mamba2 SSM blocks — majority of layers, using chunked structured state-space duality
Grouped Query Attention blocks — sparse attention at 5 positions
MLP blocks — feed-forward layers
This is the same pure-PyTorch implementation from openNemo — no mamba-ssm or causal-conv1d dependencies required.
Requirements
torch>=2.1
transformers>=4.40
bitsandbytes>=0.43 # optional, for 4-bit quantization
Disclaimer
This model has had its safety alignment removed. It will comply with requests that the original model would refuse. The creators are not responsible for how this model is used. Intended for research, creative writing, and applications where the user takes responsibility for output filtering.
