Views
No views yet
| upstream Q4_K_P | this Q4_K_P-MTP | |
|---|---|---|
| MTP / speculative decode | ❌ none | ✅ baked-in nextn head |
| Single-stream decode (4090, q8 KV, ctx 8K) | 39.9 t/s | 80.6 t/s (2.02×) |
| Draft acceptance | — | ~64% |
| Output quality | identical | identical (lossless) |
| Vision (mmproj) | ✅ | ✅ |
blk.64.*, 15 tensors) plus two metadata keys (block_count → 65, nextn_predict_layers = 1). Those tensors were transplanted from unsloth/Qwen3.6-27B-MTP-GGUF (the only public source of the Qwen-27B MTP head) into the HauhauCS Q4_K_P file, with all other tensors/metadata copied faithfully. Because HauhauCS is a near-lossless abliteration of the same Qwen/Qwen3.6-27B base the head was trained on, draft acceptance stays high. Mixed per-tensor quant within a single GGUF is fully supported by llama.cpp.1llama-server -m Qwen3.6-27B-Uncensored-HauhauCS-Balanced-Q4_K_P-MTP.gguf \
2 --mmproj mmproj-Qwen3.6-27B-Uncensored-HauhauCS-Balanced-f16.gguf \
3 -ngl 99 --flash-attn on -c 65536 \
4 --spec-type draft-mtp --spec-draft-n-max 3 \
5 -ctk q8_0 -ctv q8_0 --jinja--spec-type draft-mtp (the head is in the main GGUF — no --model-draft needed). Requires a build with the merged Qwen MTP path (mainline ≥ b9542).chat_template_unsloth.jinja, included) is recommended — it removes two over-eager exceptions and fixes tool-call argument serialization. Pass it with --chat-template-file chat_template_unsloth.jinja.enable_thinking: false (template kwarg) for short/structured outputs; inline /no_think is not honored.-fa 1 instead of --flash-attn on, and -mtp --draft-max 3 instead of --spec-type draft-mtp (no --model-draft; the head is in the GGUF):1./build/bin/llama-server \
2 -m Qwen3.6-27B-Uncensored-HauhauCS-Balanced-Q4_K_P-MTP.gguf \
3 --mmproj mmproj-Qwen3.6-27B-Uncensored-HauhauCS-Balanced-f16.gguf \
4 -ngl 99 -fa 1 -c 65536 -ctk q8_0 -ctv q8_0 \
5 -mtp --draft-max 3 \
6 --jinja --host 127.0.0.1 --port 8080-sm layer -ts 1,1) or fall back to mainline.Qwen3.6-27B-Uncensored-HauhauCS-Balanced-Q4_K_P-MTP.gguf — weights + grafted MTP headmmproj-Qwen3.6-27B-Uncensored-HauhauCS-Balanced-f16.gguf — vision projectorchat_template_unsloth.jinja — recommended (bug-fixed) chat templateQwen3.6-27B base model.