Qwen3.6-27B-Claude-Opus-Reasoning-Distilled — UD Q4_K_XL GGUF
Update 20/05/2026 — MTP draft head available: A compatible self-speculative decoding companion is now published: Qwen3.6-27B-Claude-Opus-Reasoning-MTP-Q4_K_M-gguf — load it via --model-draft for +40% decode throughput at no quality cost (llama.cpp ≥ b9245, --spec-type draft-mtp).
Just fits. For maximum reasoning quality at shorter context, see UD Q5_K_XL.
⚠️ Chat template fix required — the original rico03 repo ships the Qwen3.5 template, breaking Qwen3.6 tool calls. Add --chat-template-file qwen3.6_chat_template.jinja to your llama-server command (template source). Tool call format: <function=tool_name>{"param": "value"}</function>
Part of the DAXZEIT UD SSM-aware series — UD recipes reverse-engineered from Unsloth base models and applied to distills using calibrated imatrix.
Updated 2026-05-12 — If you downloaded this model before 12 May 2026, please re-download. The initial release used an incorrect quantization recipe. This version applies the correct Unsloth Dynamic 2.0 UD recipe with calibrated imatrix.
Despite the name, this is not a plain Q4_K_M. The effective BPW is 5.41 — higher than a standard Q5_K_M (5.00 BPW) — because the UD recipe selectively promotes critical tensors well above Q4.
Compared to a Q4_K_M vanilla (all ~498 weight tensors at Q4_K): more than half the tensors are above Q4, with the 48 SSM outputs at full Q8_0.
The 12 mid-network ffn_gate/up tensors (blk.12–16) that appear as IQ4_XS in the Unsloth base model are promoted to Q5_K in this distill — the imatrix indicates these paths are more active in the Opus reasoning fine-tune.
Perplexity
Measured on wikitext-2, full test set (~1M tokens, 1952 chunks, ctx=512), RTX 3090.
Model
PPL
BPW
Size
Q6_K plain (reference)
7.4694 ±0.031
6.57
21 GB
UD Q4_K_XL (this)
7.4712 ±0.031
5.41
17 GB
UD Q5_K_XL
7.4891 ±0.031
6.04
19 GB
UD Q6_K_XL
7.4753 ±0.031
7.64
24 GB
All four models fall within 0.02 PPL of each other — statistically equivalent (±σ overlap on all). The UD Q4_K_XL at 5.41 BPW matches the plain Q6_K at 6.57 BPW, saving 4 GB with no measurable quality loss on this benchmark.
Vision (mmproj)
This model supports vision via a multimodal projector compatible with the Qwen3.6-27B hybrid SSM+Transformer architecture.
Note: --flash-attn auto with --cache-type-k q4_0 requires llama.cpp compiled with GGML_CUDA_FA_ALL_QUANTS=ON, otherwise flash attention silently falls back to standard attention on quantized KV types.
Confirmed on RTX 3090 — 22,800 MiB / 24,576 MiB at 262K native context. Tight fit, but stable.
The UD recipe was reverse-engineered from unsloth/Qwen3.6-27B-GGUFUD-Q4_K_XL using GGUFReader tensor extraction. The architecture is identical between base model and distill, so the tensor override map transfers directly.
python
1# Extract recipe from reference GGUF2from gguf import GGUFReader
3r = GGUFReader('unsloth/Qwen3.6-27B-UD-Q4_K_XL.gguf')4for t in r.tensors:5if t.tensor_type.name notin('Q4_K','F32'):6print(f'--tensor-type {t.name}={t.tensor_type.name}')7# → 195 overrides
