Two modified versions of Qwen3.5-9B-Instruct produced by
RYS layer duplication — no training, no weight changes, just routing hidden states through a specific circuit twice.
Scores from an internal sweep benchmark run during circuit search. Sample sizes are small — treat these as directional indicators, not definitive benchmarks.
Model
Math
EQ
Reasoning
Base (32 layers)
0.563
5.5
0.235
rys_22-25 (36 layers)
0.188
0.0
0.529
rys_6-9_eq (36 layers)
0.369
24.0
0.353
Math: Ng's partial-credit scoring on a small GSM8K sample
Reasoning: fraction correct across causal, date, logic, navigation, and GSM8K probes
rys_22-25 more than doubles baseline reasoning (0.529 vs 0.235), with GSM8K rising from 0.2 to 0.8. rys_6-9_eq raises EQ from 5.5 to 24.0 and also improves reasoning above baseline, at some cost to math.
What is RYS?
Transformers self-organise during training into functional circuits — contiguous blocks of layers that act together. The RYS technique duplicates a specific block in the forward pass using the same weights, with no extra copies on disk beyond the GGUF file overhead:
The model processes the circuit twice, without any weight changes or fine-tuning.
Hybrid Mamba/attention architecture constraint
Qwen3.5-9B is a hybrid SSM/attention model (full_attention_interval = 4): full attention every 4th layer, Gated DeltaNet SSM everywhere else. The architecture repeats 8 times:
3 × (DeltaNet → FFN) → 1 × (Attention → FFN)
This creates a hard constraint on layer surgery: the total layer count must remain divisible by 4.
