Qwen3.6-27B-Claude-Opus-Reasoning-Distilled — UD Q6_K_XL GGUF
Update 20/05/2026 — MTP draft head available: A compatible self-speculative decoding companion is now published:
Qwen3.6-27B-Claude-Opus-Reasoning-MTP-Q4_K_M-gguf — load it via
--model-draft for
+40% decode throughput at no quality cost (llama.cpp ≥ b9245,
--spec-type draft-mtp).
Quantized GGUF of
rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled in
Unsloth Dynamic 2.0 Q6_K_XL format.
The highest-quality quantization in the DAXZEIT UD series for this model. Use as a quality reference or on systems with >24GB VRAM.
⚠️ Chat template fix required — the original rico03 repo ships the Qwen3.5 template, breaking Qwen3.6 tool calls. Add
--chat-template-file qwen3.6_chat_template.jinja to your llama-server command (
template source). Tool call format:
<function=tool_name>{"param": "value"}</function>
For 24GB VRAM use:
UD Q5_K_XL (131K ctx) or
UD Q4_K_XL (212K ctx).
Part of the
DAXZEIT UD SSM-aware series — UD recipes reverse-engineered from Unsloth base models and applied to distills using calibrated imatrix.
Files
| File | Size | BPW | Description |
|---|
Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q6_K_XL.gguf | 24 GB | 7.64 | UD Q6_K_XL with imatrix |
Perplexity
Measured on wikitext-2, full test set (~1M tokens, 1952 chunks, ctx=512), RTX 3090.
| Model | PPL | BPW | Size |
|---|
| Q6_K plain (reference) | 7.4694 ±0.031 | 6.57 | 21 GB |
| UD Q4_K_XL | 7.4712 ±0.031 | 5.41 | 17 GB |
| UD Q5_K_XL | 7.4891 ±0.031 | 6.04 | 19 GB |
| UD Q6_K_XL (this) | 7.4753 ±0.031 | 7.64 | 24 GB |
All four models fall within 0.02 PPL — statistically equivalent. The UD recipe achieves this by concentrating precision on the tensors that matter, making lower-BPW quants perform at the level of higher-BPW plain quants.
Tensor distribution (851 total)
| Type | Count | Tensors |
|---|
| F16 | 48 | blk.N.ssm_out.weight — SSM outputs at full F16 precision |
| Q8_0 | 134 | attn_gate, attn_k/q/v, output.weight, token_embd, late-network ffn_* |
| Q6_K | 316 | Remaining attention/FFN weights |
| F32 | 353 | Norms, biases, SSM scalars |
At Q6_K base, the ssm_out.weight tensors are kept at F16 (full 16-bit precision) rather than Q8_0 — the VRAM budget allows it and eliminates quantization error on the SSM recurrent mechanism entirely.
Vision (mmproj)
This model supports vision via a multimodal projector compatible with the Qwen3.6-27B hybrid SSM+Transformer architecture.
1llama-server \
2 -m <this-model>.gguf \
3 --mmproj Qwen3.6-27B-mmproj-hybrid-Q8_0-F16.gguf \
4 ...
Usage
1# Requires >24GB VRAM for full GPU inference
2llama-server \
3 -m Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-UD-Q6_K_XL.gguf \
4 --ctx-size 65536 \
5 --n-gpu-layers 61 \
6 --cache-type-k q4_0 \
7 --flash-attn auto \
8 --port 5000
GPU layers: benchmarked with --n-gpu-layers 61 (24GB VRAM constraint on RTX 3090 — offloads 4 layers to CPU). Use --n-gpu-layers 65 on systems with >24GB VRAM. Suitable for perplexity measurement and quality benchmarking; for production inference on 24GB use UD Q5_K_XL or UD Q4_K_XL.
Quantization Recipe
imatrix source: rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled Q6_K — wikitext-2, 200 chunks, ctx 512.
The UD recipe was reverse-engineered from
unsloth/Qwen3.6-27B-GGUF UD-Q6_K_XL using GGUFReader tensor extraction.
1from gguf import GGUFReader
2r = GGUFReader('unsloth/Qwen3.6-27B-UD-Q6_K_XL.gguf')
3for t in r.tensors:
4 if t.tensor_type.name not in ('Q6_K', 'F32'):
5 print(f'--tensor-type {t.name}={t.tensor_type.name}')
6# → 166 overrides
1llama-quantize \
2 --imatrix Qwen3.6-27B-Claude-Opus-Reasoning-Distilled.imatrix \
3 [166 --tensor-type overrides] \
4 rico03-distill-f16.gguf \
5 output-UD-Q6_K_XL.gguf \
6 Q6_K
Architecture
Qwen3.6-27B hybrid SSM+Transformer:
- 64 blocks — each block contains both SSM and attention components
- Context: 262K tokens native
- Vocabulary: 248,320 tokens
Credits
- Base model: rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled
- imatrix: generated from rico03 Q6_K on wikitext-2
- UD recipe: reverse-engineered from unsloth/Qwen3.6-27B-GGUF
- Quantization methodology: Unsloth Dynamic 2.0
- Quantized by: DAXZEIT
License
Apache 2.0 — inherited from the base model.
