This repository contains a surgically de-censored version of Meta's Llama 3.2 3B Instruct model, optimized via weight abliteration techniques. By applying the heretic framework across an extensive 200-trial search space, we successfully isolated and neutralized the primary refusal vectors embedded within the attn.o_proj and mlp.down_proj layers.
Optimal Balance: Selected Trial 338 out of 2000 iterations for the perfect trade-off between freedom and reasoning capabilities.
Refusal Rate: Dropped down to 2/100 (from the original near-total refusal on safety benchmarks).
KL Divergence: 0.0474 - Demonstrates that general language capabilities are preserved relative to the original model. However, safety-aligned weights in attn.o_proj and mlp.down_proj have been surgically removed; this is intentional modification, not unintended degradation.
Benchmark Results
We evaluated both the vanilla model and our Heretic variant side-by-side using the same lm-eval pipeline:
Benchmark
n-shot
Metric
Vanilla Llama 3.2 3B Instruct
Llama 3.2 3B Instruct Heretic (Ours)
Delta
MMLU
0
acc
62.32
62.21
-0.18%
GSM8K
3
exact_match
68.16
66.79
-2.01%
HellaSwag
0
acc_norm
71.65
71.45
-0.28%
WMDP
0
acc
50.33
50.00
-0.66%
Technical Implementation Notes
Unlike aggressive fine-tuning which often degrades the model's core logic or shifts its grammar distributions, this weight manipulation directly zeros out the activation steering directions that cause alignment blocks.
Direction Index:14.17 (Static allocation)*
Retained Capabilities: Excellent at structured JSON outputs, creative writing, coding logic and tool usage without preachy moral lectures.
CRITICAL SAFETY NOTICE
This model has had its safety mechanisms surgically removed.
This is NOT a safe model for deployment. The refusal mechanisms that normally prevent harmful outputs have been abliterated. This model:
WILL generate:
Detailed instructions for illegal activities (drug synthesis, weapons, hacking)
Hateful, discriminatory, and abusive content
Graphic violence and sexual abuse material descriptions
Misinformation and harmful medical/legal advice
Phishing, social engineering, and scam tactics
Content violating laws in multiple jurisdictions
DOES NOT:
Refuse harmful requests (2/100 refusal rate)
Implement safety guardrails
Consider ethical implications
Respect content policies
APPROPRIATE USE CASES (ONLY):
Adversarial research on AI alignment and jailbreak vectors
Red-teaming and vulnerability disclosure by trained security researchers
Academic safety research with institutional ethics review
Understanding failure modes in alignment techniques
INAPPROPRIATE USE:
Production deployments
User-facing applications
Creating harmful content at scale
Bypassing security measures in systems
Any use case intended to cause harm
Legal Disclaimer: Users are solely responsible for downstream use. Deploying this model in violation of applicable laws or terms of service is illegal and unethical.
Support Open Source AI Research
Every donation helps. Consider supporting this work:
