Qwen3.6-27B-Claude-Opus-Reasoning-Distilled — UD Q5_K_XL GGUF
Update 20/05/2026 — MTP draft head available: A compatible self-speculative decoding companion is now published:
Qwen3.6-27B-Claude-Opus-Reasoning-MTP-Q4_K_M-gguf — load it via
--model-draft for
+40% decode throughput at no quality cost (llama.cpp ≥ b9245,
--spec-type draft-mtp).
Quantized GGUF of
rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled in
Unsloth Dynamic 2.0 Q5_K_XL format.
This is the only publicly available UD Q5_K_XL quantization of this model.
⚠️ Chat template fix required — the original rico03 repo ships the Qwen3.5 template, breaking Qwen3.6 tool calls. Add
--chat-template-file qwen3.6_chat_template.jinja to your llama-server command (
template source). Tool call format:
<function=tool_name>{"param": "value"}</function>
For maximum context (212K on 24GB VRAM), see
UD Q4_K_XL. For quality reference, see
UD Q6_K_XL.
Part of the
DAXZEIT UD SSM-aware series — UD recipes reverse-engineered from Unsloth base models and applied to distills using calibrated imatrix.
Updated 2026-05-12 — If you downloaded this model before 12 May 2026, please re-download. The initial release used an incorrect quantization recipe. This version applies the correct Unsloth Dynamic 2.0 UD recipe with calibrated imatrix.
Files
| File | Size | BPW | Description |
|---|
Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q5_K_XL.gguf | 19 GB | 6.04 | UD Q5_K_XL with imatrix |
What is UD Q5_K_XL?
Unsloth Dynamic 2.0 is a mixed-precision quantization strategy that selectively upgrades the most accuracy-sensitive tensors above the base quantization level, guided by a per-tensor importance matrix (imatrix). The recipe is reverse-engineered from the official Unsloth base model GGUF and applied to the distill using a calibrated imatrix.
For Q5_K_XL the recipe uses 195 individual --tensor-type overrides on top of a Q5_K_M base:
Tensor distribution (851 total)
| Type | Count | Tensors |
|---|
| Q8_0 | 48 | blk.N.ssm_out.weight — all SSM output projections |
| Q6_K | 176 | output.weight, attn_qkv, ffn_down, attn_v, late-network ffn_gate/up |
| Q5_K | 262 | attn_gate + remaining attention/FFN weights |
| Q4_K | 12 | Mid-network ffn_gate/up (blk.12–16, imatrix-guided) |
| F32 | 353 | Norms, biases, SSM scalars |
The 48 ssm_out.weight tensors (one per block) are kept at Q8_0 — these are the output projections of the SSM recurrent mechanism, critical for long-context coherence in the Qwen3.6 hybrid architecture.
Perplexity
Measured on wikitext-2, full test set (~1M tokens, 1952 chunks, ctx=512), RTX 3090.
| Model | PPL | BPW | Size |
|---|
| Q6_K plain (reference) | 7.4694 ±0.031 | 6.57 | 21 GB |
| UD Q4_K_XL | 7.4712 ±0.031 | 5.41 | 17 GB |
| UD Q5_K_XL (this) | 7.4891 ±0.031 | 6.04 | 19 GB |
| UD Q6_K_XL | 7.4753 ±0.031 | 7.64 | 24 GB |
All four models fall within 0.02 PPL of each other — statistically equivalent (±σ overlap on all). The UD recipe concentrates precision on the tensors that matter most, achieving Q6_K-level quality at a lower bit-per-weight budget.
Vision (mmproj)
This model supports vision via a multimodal projector compatible with the Qwen3.6-27B hybrid SSM+Transformer architecture.
1llama-server \
2 -m <this-model>.gguf \
3 --mmproj Qwen3.6-27B-mmproj-hybrid-Q8_0-F16.gguf \
4 ...
Usage
1llama-server \
2 -m Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q5_K_XL.gguf \
3 --mmproj Qwen3.6-27B-mmproj-Q8_0.gguf \
4 --ctx-size 131072 \
5 --n-gpu-layers 65 \
6 --cache-type-k q4_0 \
7 --cache-type-v q4_0 \
8 --flash-attn auto \
9 --port 5000
Fits in 24GB VRAM at 65 GPU layers, Q4_0 KV cache, 131K context with mmproj.
Quantization Recipe
imatrix source: rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled Q6_K — wikitext-2, 200 chunks, ctx 512.
The UD recipe was reverse-engineered from
unsloth/Qwen3.6-27B-GGUF UD-Q5_K_XL using GGUFReader tensor extraction. The architecture is identical between base model and distill, so the tensor override map transfers directly.
1# Extract recipe from reference GGUF
2from gguf import GGUFReader
3r = GGUFReader('unsloth/Qwen3.6-27B-UD-Q5_K_XL.gguf')
4for t in r.tensors:
5 if t.tensor_type.name not in ('Q5_K', 'F32'):
6 print(f'--tensor-type {t.name}={t.tensor_type.name}')
7# → 195 overrides
1llama-quantize \
2 --imatrix Qwen3.6-27B-Claude-Opus-Reasoning-Distilled.imatrix \
3 [195 --tensor-type overrides] \
4 rico03-distill-f16.gguf \
5 output-UD-Q5_K_XL.gguf \
6 Q5_K_M
Architecture
Qwen3.6-27B hybrid SSM+Transformer:
- 64 blocks — each block contains both SSM and attention components
- Context: 262K tokens native
- Vocabulary: 248,320 tokens
Credits
- Base model: rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled
- imatrix: generated from rico03 Q6_K on wikitext-2
- UD recipe: reverse-engineered from unsloth/Qwen3.6-27B-GGUF
- Quantization methodology: Unsloth Dynamic 2.0
- Quantized by: DAXZEIT
License
Apache 2.0 — inherited from the base model.
