Qwen3.6-27B-Claude-Opus-Reasoning-Distilled — RA Series (Reasoning-Aware Quantization)
Update 20/05/2026 — MTP draft head available: A compatible self-speculative decoding companion is now published: Qwen3.6-27B-Claude-Opus-Reasoning-MTP-Q4_K_M-gguf — load it via --model-draft for +40% decode throughput at no quality cost (llama.cpp ≥ b9245, --spec-type draft-mtp).
RA is a domain-specific quantization methodology that allocates precision based on tensor importance during reasoning tasks, not generic text.
The distillation transferred something specific from Claude Opus — a way of structuring reasoning, holding uncertainty, working through problems step by step. Standard quantization doesn't care about any of that: it calibrates on generic text and compresses everything equally. I wanted to see whether allocating precision according to what the model actually does during reasoning could preserve that distilled behavior better. The results are modest but consistent (−0.0010 to −0.0014 PPL on reasoning across all three levels). I run RA3 daily as the backbone of a 30-tool agent scaffold, and it holds coherence on long sessions in ways I haven't seen from uniform quants at the same size — but that's anecdote, not evidence. If you test these quants, I'd genuinely like to know what you find.
Why RA?
Standard quantization (bartowski, mradermacher) calibrates imatrix on generic text — wikitext or similar. This tells you which tensors are important for general text continuation. For a reasoning-distilled model, this averages out the signal: tensors critical for reasoning get the same precision as tensors that are barely active during reasoning.
RA calibrates on the model's own training data — 14K Opus reasoning traces, the same data used to distill the model. By comparing per-tensor importance between the reasoning imatrix and the wikitext imatrix, we identify which tensors are specifically activated by the reasoning domain.
The ratio reveals a clear pattern:
Tensor Region
Reasoning/Wikitext Ratio
Interpretation
blk.12–16 (mid-network)
0.66–0.76
Less active during reasoning — demote
blk.49–53 (late-network)
1.09–1.12
More active during reasoning — promote
blk.63.ffn_down
58,620 importance
Most critical tensor — 5× the second-ranked
The delta is the signal. Precision follows activation, not convention.
Three Variants
Each RA level increases precision allocation on reasoning-critical tensors. The progression tells you where the ceiling is.
Benchmarks
Variant
BPW
Size
PPL wikitext
PPL reasoning
F16 tensors
Q4_K_XL (generic)
5.41
17.0 GB
6.8341
2.6839
0
RA1_XL
5.47
17.1 GB
6.8421
2.6829
0
RA2_XL
5.56
17.4 GB
6.8446
2.6827
0
RA3_XL
5.69
18.0 GB
6.8411
2.6825
7
Q6_K plain (generic)
6.57
21.0 GB
baseline
baseline
0
RA3 at 5.69 BPW / 18 GB matches or beats Q6_K plain at 6.57 BPW / 21 GB — with 1.16 fewer bits per weight.
RA4 was tested (all 48 SSM outputs promoted from Q8_0 to F16, ~6.36 BPW / 19.4 GB): identical PPL to RA3. The precision ceiling was already reached at RA3. The negative result is documented — data tells you when to stop.
RA1 — Reasoning Swap (first allocation)
Swap precision between underactive and overactive regions. Budget-neutral.
Recipe:
Demote blk.12–16 ffn_gate/up → Q4_K (freed budget from low-activation tensors)
Promote blk.49–53 ffn_gate/up → Q6_K (invest in high-activation tensors)
Who is this for: Maximum context window. Minimal VRAM overhead. First validation that domain-specific calibration produces a signal.
MTP: RA1 is the recommended variant for self-speculative decoding. Its lower VRAM footprint (~17.1 GB) leaves headroom for the MTP draft head (+1.9 GB) and KV cache without exceeding 24 GB — measured total: ~23.2 GB on RTX 3090 at 120K context.
RA2 — Reasoning Boost (top-12 promoted)
Promote top-12 reasoning tensors from imatrix delta to Q8_0.
Compare the two imatrices. For each tensor, compute:
ratio = importance_reasoning / importance_wikitext
Ratio > 1.0 → tensor is more active during domain tasks → promote
Ratio < 1.0 → tensor is less active during domain tasks → candidate for demotion
Step 3: Apply RA levels
Start with RA1 (budget-neutral swap), measure PPL on both wikitext and domain data. If PPL holds, proceed to RA2, then RA3. Stop when PPL stops improving between levels — that's your ceiling.
Step 4: Verify
Each quantization script includes a verification step that counts tensor types and confirms F16/Q8_0 placement matches the recipe.
