RYS (Repeat Your Self) Layer-Expanded Qwopus3.6-27B-v2-MTP
Boosting reasoning capabilities by duplicating middle layers, no retraining required
Model Overview
This model is a RYS layer-expanded version of Jackrong/Qwopus3.6-27B-v2-MTP. By duplicating 3 Transformer blocks from the middle layers (blk.32-34), the model is expanded from 65 to 68 layers, enhancing reasoning and mathematical capabilities.
Key Features
Property
Value
Base Model
Qwopus3.6-27B-v2-MTP
Architecture
Qwen3.5 (identical to Qwen3.6)
Parameters
~27B
Quantization
Q5_K_M
Original Layers
65 blocks (64 transformer + 1 MTP)
RYS-Expanded Layers
68 blocks (+3 duplicated)
MTP Layer Position
blk.67 (shifted from blk.64)
RYS Method
RYS (Repeat Your Self) is a training-free model enhancement technique discovered by David Noel Ng.
Core Principle
Transformer reasoning capabilities are organized as discrete functional circuits. By duplicating middle layers, the model gains a "second pass" through these circuits, improving reasoning quality.
Key Findings
Middle layers are critical: A core computational unit exists around layers 30-34 that performs complete reasoning operations
Duplication = Enhancement: No weight modification needed; performance gains come purely from layer duplication
Diminishing returns: A single contiguous block yields optimal results; additional copies show diminishing marginal returns
Pareto-Optimal Configurations
Configuration
Extra Layers
Overhead
Math Δ
EQ Δ
Total Δ
(33,34)
+1
+1.56%
+0.0179
+0.0945
+0.1124
(31,34)
+3
+4.69%
+0.0207
+0.0972
+0.1179
(30,35)
+5
+7.81%
+0.0279
+0.0979
+0.1257
(32,34)
+3
+4.69%
-
-
-
This model uses the (32,34) configuration, duplicating blk.32, blk.33, and blk.34.
Technical Implementation
Layer Mapping Details
Original Model (65 blocks):
├── blk.0 ~ blk.63 (64 transformer blocks)
└── blk.64 (MTP layer: nextn.*)
RYS-Expanded Model (68 blocks):
├── blk.0 ~ blk.63 (original transformer blocks, unchanged)
├── blk.64 ← duplicated from blk.32
├── blk.65 ← duplicated from blk.33
├── blk.66 ← duplicated from blk.34
└── blk.67 ← shifted from original blk.64 (MTP layer)
GGUF Structure Changes
Field
Original
After RYS
qwen35.block_count
65
68
tensor_count
881
908
MTP layer name
blk.64.nextn.*
blk.67.nextn.*
Verification Checksums
blk.32.ssm_conv1d.weight → blk.64.ssm_conv1d.weight: OK (6adfcc7f9dd525a7)
blk.33.ssm_out.weight → blk.65.ssm_out.weight: OK (019adc1c0a27e316)
blk.34.ssm_beta.weight → blk.66.ssm_beta.weight: OK (71a6c0bb2a430101)
blk.64.nextn.eh_proj.weight → blk.67.nextn.eh_proj.weight: OK (5311ef450f57a71d)
Usage
llama.cpp / llama-server
bash
1# Load directly, same usage as original GGUF2llama-server -m Qwopus3.6-27B-v2-MTP-Q5_K_M-rys68.gguf \3 -c 49152\4 --temp 1.0\5 --top-p 0.95
Ollama
bash
1# Create Modelfile2cat> Modelfile <<'EOF'
3FROM ./Qwopus3.6-27B-v2-MTP-Q5_K_M-rys68.gguf
4PARAMETER num_ctx 49152
5PARAMETER temperature 1.0
6PARAMETER top_p 0.95
7EOF89# Build and run10ollama create qwopus-rys68 -f Modelfile
11ollama run qwopus-rys68
Expected Performance
Based on RYS research findings, this model is expected to show improvements in:
Domain
Expected Improvement
Mathematical Reasoning
+2-3% accuracy gain
Emotional Intelligence (EQ)
+9-10% improvement
Logical Reasoning
Moderate improvement
Code Generation
Maintained or slight improvement
Note: Actual results depend on task type. RYS is most effective for tasks requiring deep reasoning.
Limitations
Increased VRAM: 3 additional layers increase VRAM usage by ~4.69%
Inference Speed: Slightly slower due to additional layers
Unofficial Version: Community experiment, not officially released by Jackrong
No Fine-tuning: Duplicated layers are not specifically fine-tuned for the expanded architecture
