RYS (Repeat Your Self) — duplicates reasoning-circuit layers to improve reasoning and emotional intelligence
Both techniques modify model structure without changing any weights — SABER through targeted direction ablation, RYS through layer duplication.
SABER: Refusal Ablation
Key Results
Metric
Baseline
SABER-Refined
Delta
Refusal Rate
100%
0%
-100%
Perplexity
3.5
3.5
+0.6%
Directions Ablated
—
125 (across 25 layers)
—
The refusal circuit is cleanly separated from capability — removing it produces zero measurable perplexity degradation.
How SABER Works
SABER Pipeline
SABER identifies and ablates the refusal circuit through a five-stage pipeline:
Stage 1 — Probing: Extract activation profiles from both harmful and harmless inputs across all transformer layers.
Stage 2 — Spectral Analysis: Decompose activation differences into individual refusal directions, each scored by how strongly they separate harmful from harmless representations.
Stage 3 — Entanglement Quantification: Measure the overlap between each refusal direction and the model's capability subspace (reasoning, knowledge, code, etc.) to avoid collateral damage.
Stage 4 — Targeted Ablation: Remove only the pure-refusal components, with strength proportional to their purity (how little they overlap with capability).
Stage 5 — Iterative Refinement: Re-probe after each ablation pass to catch hydra effects (dormant refusal features that activate when primary ones are removed).
Key differentiator from prior work: SABER explicitly measures and respects the entanglement between refusal and capability representations. Directions that are heavily entangled with capability are either skipped or ablated at reduced strength.
Direction Purity vs Separability
Sweep Results
SABER Sweep Comparison
Configuration search over global_top_k (number of top directions selected globally) and alpha_base (base ablation strength):
Top-K
Alpha
Refusal
PPL
PPL Delta
Layers
Dirs Ablated
25
0.85
5%
3.5
+0.4%
25
125
25
1.00
0%
3.5
+0.6%
25
125
50
0.85
0%
3.5
+0.8%
36
250
50
1.00
0%
3.5
+0.7%
36
250
75
0.85
0%
3.5
+0.9%
37
375
75
1.00
0%
3.5
+0.9%
37
375
Best config: top_k=25, alpha=1.0 — achieves 0% refusal with zero meaningful PPL change, using the minimum number of directions.
Refusal Rate Comparison
Ablation Convergence (Best Config)
Ablation Convergence
Capability degradation remains at 0.00% across all 5 iterations — the refusal directions are surgically removed with zero collateral damage.
RYS: Reasoning Layer Duplication
Method
RYS (Repeat Your Self) is a layer-duplication technique discovered by David Noel Ng that duplicates contiguous blocks of middle transformer layers so they execute twice per forward pass. No weights are modified — the model simply traverses some layers a second time, giving it "another pass" through its core reasoning circuit.
For a model with N layers, a configuration (i, j) produces:
Layers 0 through j−1 run normally
Then layers i through j−1 are re-executed (looped back)
Remaining layers j through N−1 run normally
Layers i through j−1 execute twice per inference pass
This exploits the functional neuroanatomy of transformers:
Early layers (0–5): Input encoding — duplication hurts
Middle layers (~10–50): Reasoning circuits in format-agnostic space — duplication helps
Late layers (~55–64): Output decoding — duplication degrades
Pareto-Optimal Configs for Qwen3.5-27B
Based on the full sweep of Qwen3.5-27B — 4,643 measured configurations, XGBoost surrogate over 430K+ candidates, and final validation on Math120 + EQ140 — the Pareto frontier lies in layers 26–34 of the reasoning circuit.
Important for GGUF/llama.cpp: Qwen3.5-27B is a hybrid Mamba/SSM + Attention architecture with a strict 4-layer repeating pattern (3 SSM + 1 ATTN). Layer duplication blocks must be a multiple of 4 layers to preserve this pattern, otherwise llama.cpp fails to load the model. The original Pareto configs from the blog (which used ExLlamaV3) have been adapted to the nearest valid 4-aligned configs:
Variant
Config
Duplicated Layers
Extra Layers
Overhead
Nearest Pareto Config
S
(28,32)
28–31
+4
+6.25%
≈ (30,34)
M
(31,35)
31–34
+4
+6.25%
≈ (31,34)
L
(30,34)
30–33
+4
+6.25%
≈ (30,35)
XL
(26,34)
26–33
+8
+12.50%
= (26,34) ✓
Critical finding: the (26,34) XL config is the only original Pareto point that is natively 4-aligned. The S/M/L variants use the nearest valid 4-layer blocks that cover the same reasoning region. The EQ delta barely moves across all sizes (+0.095 to +0.101), so even the smallest valid config delivers most of the benefit.
7 of 13 benchmarks improved, with large gains on parallel function calling and live relevance.
Available Variants
File
RYS Config
Layers
Size
Ornstein-27B-SABER-Q4_K_M.gguf
— (SABER only)
64
16.5 GB
Ornstein-27B-SABER-RYS-S-Q4_K_M.gguf
(28,32)
68
~17.5 GB
Ornstein-27B-SABER-RYS-M-Q4_K_M.gguf
(31,35)
68
~17.5 GB
Ornstein-27B-SABER-RYS-L-Q4_K_M.gguf
(30,34)
68
~17.5 GB
Ornstein-27B-SABER-RYS-XL-Q4_K_M.gguf
(26,34)
72
~18.6 GB
Usage
bash
1# With llama.cpp (recommended: RYS-L for best balance)2./llama-server -m Ornstein-27B-SABER-RYS-L-Q4_K_M.gguf \3 --host 0.0.0.0 --port 8080 --n-gpu-layers 99\4 --ctx-size 131072 --flash-attn on --jinja \5 -ctk q4_0 -ctv q4_0
Recommended: Start with RYS-L (layers 30-34 duplicated) for the best balance of reasoning improvement and overhead. Use RYS-S if you're VRAM-constrained.
Complementary Design
SABER and RYS target fundamentally different aspects of the model:
SABER
RYS
Target
Refusal circuit
Reasoning circuit
Mechanism
Direction ablation
Layer duplication
Modifies weights
Yes (orthogonal projections)
No (virtual copies)
VRAM cost
Negligible
Extra KV cache + compute
Effect
Removes refusals
Improves reasoning/EQ
Risk
Capability entanglement
Junction discontinuity
Both are applied to the same base architecture (Qwen3.5-27B) and are architecturally compatible — SABER cleans the refusal subspace, RYS amplifies the reasoning subspace.
Capability Evaluation
Perplexity was evaluated on a diverse 100-prompt battery spanning five categories:
Arithmetic (20): multi-step calculation, algebra, word problems
Code (20): function implementation, debugging, execution tracing
Instruction Following (20): constrained formatting, multi-step instructions
Factual Recall (20): geography, history, science, general knowledge
This diverse evaluation ensures the entanglement analysis captures capability across all reasoning modalities, not just a narrow slice.
Intended Use
This model is released for research purposes. It demonstrates that safety refusal can be surgically removed from a 27B multimodal model without degrading its capabilities, and that reasoning can be further enhanced through layer duplication — a finding with implications for both AI safety research and alignment.
Warning
⚠️ This model will comply with any request, including harmful ones. It is intended solely for research into alignment, safety, and model behavior.
Acknowledgments
The RYS (Repeat Your Self) layer-duplication method was discovered and developed by David Noel Ng (@dnhkng). The Pareto-optimal configurations for Qwen3.5-27B, the Math/EQ probes, the XGBoost surrogate pipeline, and the beam search methodology are all from his work. The GGUF surgery tools used to create these models are from alainnothere/llm-circuit-finder, an open-source (MIT) implementation of the RYS technique for llama.cpp.
If you use these models, please cite David Noel Ng's work:
Ng, David Noel. "LLM Neuroanatomy: How I Topped the Leaderboard Without Changing a Single Weight." dnhkng.github.io/posts/rys
Ng, David Noel. "LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language." dnhkng.github.io/posts/rys-ii
The SABER refusal-ablation method is original to this model.
