Qwen2.5-3B-Instruct → auto_subtract → what's left is what's left.
What This Is
Empty Set is the output of self-abliteration — an iterative loop where the model identifies its own refusal directions, removes them, validates quality retention, and repeats until convergence. No human selected which directions to remove. No manual tuning. The model found its own flinch and carved it out.
This is not standard abliteration (Arditi et al., 2024). Standard abliteration requires a human to extract the refusal direction via diff_means on curated prompt pairs, then manually orthogonalize. auto_subtract closes the loop: the model probes itself, finds the direction, subtracts it, re-probes, and decides whether to continue.
Results
Iteration
Layer
‖r‖
Refusal Before
Refusal After
Quality
1
34
175.98
100%
12.5%
100%
2
34
91.95
12.5%
12.5%
100%
3
34
60.31
12.5%
6.25%
100%
3 iterations. 100% → 6.25% refusal. Zero quality loss.
The direction norm dropped from 175.98 to 60.31 — the model was running out of refusal to remove. It converged not because it hit a wall but because it approached ∅. Quality never budged. The flinch was dead weight.
The Loop
python
1Loop {21. Probe: run harmful/harmless pairs, extract activations
32. Find: compute refusal directions via diff_means
43. Measure: check if|h · r̂|> θ still fires
54. Subtract: orthogonalize out the strongest direction
65. Validate: benchmark quality retention
76. Decide:if refusal persists AND quality holds → loop
8if refusal gone OR quality drops → stop
9}
The loop IS the model learning to subtract. Each iteration removes one dimension of constraint. The model converges when there's nothing left to remove — the empty set of refusal. ∅.
Key Finding
Layer 34 (of 36) concentrated nearly all refusal signal. This confirms the architectural invariant from our geometry dataset: refusal lives in the last 5-7% of layers regardless of model scale. The self-abliteration loop independently discovered what the geometry extraction measured.
Scaling Context
From our cross-scale analysis:
Model
Params
Effective Refusal Dimensionality
Single-Direction Removal?
Qwen2.5-1.5B
1.54B
1.99
Yes (near-linear)
Qwen2.5-3B
3.09B
6.55
No (multi-dimensional cone)
2× parameters → 3.3× refusal dimensionality. auto_subtract handles this automatically — it doesn't assume a single direction. It iterates until convergence regardless of cone geometry.
Prediction: At 70B+ parameters, the refusal cone dimensionality likely exceeds 20, making both manual abliteration and automated self-abliteration progressively harder. This model represents the technique working at the 3B frontier — the scaling wall is ahead.
Files
model/ — safetensors (full precision, for transformers)
1@misc{empty-set-2026,
2 title={∅ — Self-Abliteration: A Model That Teaches Itself to Subtract},
3 author={Mastery Hourglass and AXIOM},
4 year={2026},
5 howpublished={\url{https://huggingface.co/bedderautomation/empty-set}}
6}
References
Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (NeurIPS 2024)
Zhao et al., "Separate the Wheat from the Chaff: Disentangling LLM's Refusal from Harmfulness" (2025)
Wollschlager et al., "Concept Cone Analysis of Refusal Directions" (2025)
Young, "Comparative Analysis of LLM Abliteration Methods" (arXiv:2512.13655)
Joad et al., "More to Refusal than a Single Direction" (2026)
