This is the first GGUF of Qwopus3.6-27B that preserves the MTP (Multi-Token Prediction) draft head, enabling speculative decoding for ~2x generation speedup. No other published GGUF of this model includes MTP.

What's in this model

Layer	Source	What it does
Base weights	Jackrong/Qwopus3.6-27B-v1-preview	Claude 4.6 Opus reasoning distillation via SFT on Qwen3.6-27B. Structured chain-of-thought, reduced reasoning variance, more consistent outputs.
Abliteration	croll83/Qwopus3.6-27B-v1-Abliterated-preview	Qwen-layer refusal direction removed in activation space. See abliteration caveat below.
MTP draft head	Qwen/Qwen3.6-27B-FP8	Vanilla Qwen3.6-27B MTP head grafted onto Qwopus weights. FP8 block-scaled weights dequantized to BF16 before grafting.

⚠️ Abliteration caveat: partial refusal behavior

This model retains some refusal behavior despite abliteration. This is not a bug in the abliteration process — it is an inherent consequence of Claude-distilled reasoning.

croll83's abliteration successfully removes Qwen's base refusal direction at the weight level. However, Jackrong's Qwopus fine-tune was trained on Claude 4.6 Opus reasoning traces via SFT. Claude's safety evaluation logic is embedded directly in those reasoning traces — the model learned to think through safety decisions as part of learning Claude's reasoning style.

In practice, this means the model will sometimes refuse sensitive requests (e.g., exploit development, certain cybersecurity topics) by reasoning through Claude's safety framework in its <think> block, even though the Qwen-layer refusal vector has been removed. We verified this behavior exists in both our MTP-grafted model and croll83's base abliterated checkpoint — confirming it originates from the Qwopus SFT data, not from the MTP graft or quantization process.

This is a novel finding: Claude's safety properties survive abliteration when transferred via reasoning trace distillation, because the safety behavior is encoded in the model's learned reasoning patterns rather than in a removable refusal direction vector. Abliteration targets the latter but cannot reach the former.

If you need a fully uncensored model, use a non-Claude-distilled abliterated Qwen3.6-27B variant (e.g., huihui-ai/Huihui-Qwen3.6-27B-abliterated or the heretic-v2 NVFP4 GGUFs). The trade-off is losing the Claude-style structured reasoning in exchange for full uncensoring.

Available quantizations

File	Size	BPW	Use case
`Qwopus3.6-27B-Abliterated-MTP-Q8_0.gguf`	28 GB	8.50	Best quality. RTX Pro 6000, RTX 5090, any 32GB+ GPU.
`Qwopus3.6-27B-Abliterated-MTP-Q4_K_M.gguf`	16 GB	4.92	Great quality. RTX 3090, 4090, 5070 Ti, any 24GB GPU.

Benchmark results (RTX Pro 6000 Blackwell, 96GB VRAM)

Tested with llama-server build b9281+, --spec-type draft-mtp --spec-draft-n-max 3.

Q8_0 results

Metric	Value
Decode (first request)	86–110 tok/s
Decode (sustained, long output)	80–99 tok/s
Prefill (cold)	268 tok/s
Prefill (conversation reprocess)	2,084–2,378 tok/s
MTP acceptance rate	44–68%

Baseline comparison (same hardware, Qwen3.6-27B heretic-v2 MTP GGUF)

Metric	Heretic-v2 MTP	Qwopus MTP (this model)
Decode	93–110 tok/s	80–99 tok/s
MTP acceptance	61–76%	44–68%
Reasoning style	Vanilla Qwen	Claude-structured

The lower MTP acceptance rate is expected — the grafted draft head was trained against vanilla Qwen3.6-27B hidden states, and Qwopus's fine-tune shifts the activation space. The net throughput is still approximately 2x the ~42 tok/s non-MTP baseline.

Recommended serve commands

Best config (MTP + prompt caching)

The --no-context-shift flag is critical for enabling prompt caching on hybrid GDN models. Without it, llama.cpp's context shift validation logic invalidates GDN recurrent state checkpoints on every turn, forcing full prompt reprocessing. See the detailed findings below.

bash

llama-server \
  -m Qwopus3.6-27B-Abliterated-MTP-Q8_0.gguf \
  -ngl 99 -c 262144 -fa on -np 1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  --no-context-shift \
  --host 0.0.0.0 \
  --port 8000

For 24GB GPUs (Q4_K_M)

bash

llama-server \
  -m Qwopus3.6-27B-Abliterated-MTP-Q4_K_M.gguf \
  -ngl 99 -c 32768 -fa on -np 1 \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --spec-type draft-mtp --spec-draft-n-max 1 \
  --no-context-shift \
  --host 0.0.0.0 \
  --port 8000

⚠️ Correction: `--cache-ram` removed from recommended configs

Previous versions of this card recommended --cache-ram 49152. This was a mistake based on a misunderstanding of what the flag does.

--cache-ram does NOT put the KV cache in VRAM. It allocates system RAM as a host-memory prompt snapshot cache (llama.cpp PR #16391 by Ggerganov). When a request finishes, llama-server saves the computed KV state to system RAM so it can be hot-swapped back if a similar prompt prefix appears. At 262K context with a large prompt, a single snapshot can exceed 12 GB of system RAM. On a 64GB system running WSL2, this caused an OOM kill — the kernel terminated llama-server at 52.5GB of anon-rss usage.

The KV cache itself lives in VRAM regardless of this flag. For single-user setups with --parallel 1, the VRAM KV cache already persists between requests on the same slot, making --cache-ram redundant. The flag is only useful for multi-slot servers where slot eviction is frequent.

If you previously set --cache-ram and experienced OOM crashes, remove it. Your KV cache is already in VRAM where it belongs.

Note on --no-context-shift: This disables the automatic context window rotation that discards old tokens when the context fills up. With 262K context, you are unlikely to fill the window in normal use. The trade-off is that generation stops if you ever hit the limit instead of silently dropping early context.

For coding: This is a non-issue. Agentic coding harnesses (Qwen Code, OpenCode, Claude Code, aider, etc.) have built-in compaction layers that typically reserve a ~30% buffer — meaning they automatically summarize and trim conversation history at ~70% context usage (~183K tokens), well before you ever reach the hard limit. You will never have the model stop generating due to this flag.

For chatting: Most conversations will never approach 262K tokens. If you are running extremely long sessions, check whether your chat frontend supports manual or automatic compaction. Some interfaces (like OpenWebUI) do not compact automatically, so in theory you could hit the limit during a very extended conversation — but at 262K tokens (~200,000 words), that would require many hours of continuous dialogue. If your chat frontend does support compaction, this flag becomes a no-op for free performance.

Bottom line: This is a free performance flag with no practical downside for the vast majority of use cases. Use it until the upstream GDN prompt caching bug is fixed, at which point it becomes unnecessary.

Prompt caching on hybrid GDN models — findings and workarounds

Qwen3.6 uses a hybrid architecture with 48 Gated DeltaNet (GDN) layers and 16 full attention layers. The GDN layers maintain a recurrent state that interacts poorly with prompt caching in both llama.cpp and vLLM. This affects ALL Qwen3.6-27B variants, not just this model.

Root cause: two bugs in llama.cpp (as of build bb28c1fe2)

Through extensive debugging, we identified two specific bugs in llama.cpp that cause checkpoint invalidation on hybrid/recurrent models. Neither has been merged upstream as of May 2026.

Bug 1: llama_kv_cache_iswa::get_can_shift() size equality check

In src/llama-kv-cache-iswa.cpp, the function checks kv_base->get_size() == kv_swa->get_size(). For hybrid models like Qwen 3.6, the base and SWA caches are intentionally different sizes (different layer counts for attention vs GDN). This check always returns false, permanently blocking the cache reuse path.

Fix: Remove the size equality check. The individual get_can_shift() calls on each sub-cache already guard shift safety independently.

cpp

// BEFORE (broken for hybrid models):
bool llama_kv_cache_iswa::get_can_shift() const {
    return kv_base->get_can_shift() &&
           kv_swa->get_can_shift() &&
           kv_base->get_size() == kv_swa->get_size();
}

// AFTER (fixed):
bool llama_kv_cache_iswa::get_can_shift() const {
    return kv_base->get_can_shift() &&
           kv_swa->get_can_shift();
}

This fix was independently identified by sujitvasanth and verified on Gemma 4 and Qwen 3.5 MoE models. It applies to all ISWA (Interleaved Sliding Window Attention) models.

Bug 2: Checkpoint search uses SWA-based pos_min threshold that fails for GDN models

In tools/server/server-context.cpp, the checkpoint search lambda (line ~2694) uses cur.pos_min < pos_min_thold to find a valid checkpoint. For hybrid models with n_swa = 0, pos_min always equals the full sequence length, so no checkpoint is ever matched. Additionally, the checkpoint creation threshold requires n_tokens >= 64, which prevents early checkpoints for short prompts.

Fix: Add hybrid model detection and use pos_max-based matching instead:

cpp

// In the checkpoint search lambda (~line 2694):
if (llama_model_is_recurrent(model_tgt) || llama_model_is_hybrid(model_tgt)) {
    return cur.pos_max <= pos_next;
}
return cur.pos_min < pos_min_thold || cur.pos_min == 0;

// In the checkpoint creation threshold (~line 2931):
const int checkpoint_min_tokens =
    (llama_model_is_recurrent(model_tgt) || llama_model_is_hybrid(model_tgt)) ? 4 : 64;
do_checkpoint = do_checkpoint && (pos_min >= 0 && slot.prompt.n_tokens() >= checkpoint_min_tokens);

These fixes were developed and tested on build bb28c1fe2 against Qwen3.6-27B (this model) and the DavidAU Qwen3.6-40B Deckard expanded model.

Verified result with both patches applied: Checkpoint restoration works reliably across turns. On the 40B Deckard at 85K+ tokens context, checkpoints restore in ~20ms and only new tokens are processed each turn. On this model (27B with MTP), the same behavior is observed with --no-context-shift enabled.

The `--no-context-shift` workaround

Adding --no-context-shift to llama-server enables prompt caching to work on hybrid GDN models even without the source patches above. Without this flag, the context shift validation logic checks checkpoint positions against SWA-related thresholds, and the GDN recurrent state's pos_min always exceeds these thresholds, causing llama.cpp to invalidate checkpoints and force full reprocessing every turn.

If you apply both source patches described above, --no-context-shift becomes less critical but is still recommended as a belt-and-suspenders measure until the fixes are merged upstream.

Frontend-specific behavior (tested with `--no-context-shift` enabled)

llama.cpp built-in UI (http://localhost:8000):

Prompt caching works immediately from the second turn onward
Logs show restored context checkpoint on follow-up messages
Only new tokens are processed on each turn; cached prefix is reused

OpenWebUI (via Docker, OpenAI-compatible API):

First 2–3 turns still show forcing full prompt re-processing
After the conversation accumulates enough tokens (~2K+), caching begins working
Logs show restored context checkpoint on later turns
Likely cause: OpenWebUI injects metadata or formats requests slightly differently than raw API calls, causing checkpoint position misalignment on early turns. Once the conversation grows past the checkpoint boundaries, the offsets become negligible and the cache hits
At 2,000+ tok/s prefill on Blackwell, the uncached early turns take under 1 second and are not noticeable

curl (isolated requests):

Cross-conversation prefix sharing works immediately (e.g., two separate requests with the same system prompt)
Multi-turn simulation via curl with conversation history appended: cache restores correctly with --no-context-shift

vLLM findings (updated May 2026)

vLLM prefix caching DOES work for Qwen3.6 — but only without MTP.

Previous versions of this card stated that vLLM prefix caching was 0% across the board. After further testing, we found:

Configuration	Prefix cache hit rate	Notes
Qwen3.6-27B-AEON-FP8-MTP on vLLM, with MTP	91.1%	Climbed from 0% → 47.8% → 72.2% → 86.6% → 91.1% over ~10 turns
Qwen3.6-27B-FP8 on vLLM, no MTP	~15.7%	Baseline, curl-tested
Qwen3.6-27B on vLLM, with MTP	0%	MTP kills prefix caching on some model variants

The key finding: vLLM's prefix caching works well for Qwen3.6 hybrid models when the model weights and MTP head are properly aligned (as in the AEON FP8-MTP checkpoint). The 91.1% hit rate was achieved with:

bash

vllm serve Qwen3.6-27B-AEON-FP8-MTP \
  --port 8080 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --max-num-seqs 2 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

Generation throughput on vLLM was 72–80 tok/s with 65–82% MTP acceptance. This is a viable alternative to llama.cpp for users who want prefix caching without source patches, though llama.cpp with the patches applied provides faster per-token generation and better checkpoint granularity.

vLLM does NOT work well for expanded/non-standard architectures. The DavidAU 40B Deckard (96-layer expanded model) ran at 4–11 tok/s on vLLM due to unoptimized Triton GDN kernels for the non-standard layer count. The same model runs at 32–35 tok/s on llama.cpp with the fused GDN CUDA kernel.

What does NOT work

llama.cpp WITHOUT --no-context-shift (and without source patches):

Multi-turn caching fails on every turn regardless of frontend
The GDN recurrent state checkpoint positions from the previous turn's generation don't align with the new request's expected positions
Exception: the built-in llama.cpp UI sometimes restores checkpoints even without the flag, but this is inconsistent

llama.cpp with Unsloth UD-Q8_K_XL GGUF (non-heretic/non-NVFP4 base):

MTP with --spec-draft-n-max 3 breaks prompt caching even with --no-context-shift
--spec-draft-n-max 1 preserves caching on this GGUF variant
The heretic NVFP4-MLP-Only-Q8_0 and this Qwopus model work with --spec-draft-n-max 3
This behavior difference across GGUF variants is not documented elsewhere as of May 2026

Upstream issues

llama.cpp #13194 — kv-cache SWA support (source of the checkpoint validation logic)
llama.cpp #20225 — Full prompt re-processing on every turn for hybrid models
llama.cpp #22384 — Root cause analysis and one-line fix for ISWA get_can_shift() (sujitvasanth)
llama.cpp #22534 — PR to prevent destructive memory wipe on seq_rm failure for hybrid models (spiritbuun, not merged)
llama.cpp #22746 — Qwen3.6-27B specific
llama.cpp #22940 — PR to permanently disable prompt cache for FULL-only memory backends (Open)
llama.cpp #23013 — Qwen3.6-27B specific
vllm #36493 — Prefix caching near 0% on Qwen3.5 hybrid models
vllm #38182 — MTP reduces prefix cache hit rate on Qwen3.5

MTP graft methodology

The Qwopus fine-tune (and croll83's abliteration) dropped the MTP draft head tensors. We grafted vanilla Qwen3.6-27B's MTP head back in using the pipeline below. This technique was pioneered by kasimat for the AEON model.

1. Dequantize the FP8 MTP shard

The MTP shard from Qwen/Qwen3.6-27B-FP8 uses FP8 (float8_e4m3fn) with block-scaled quantization (128×128 blocks). Each weight tensor has a corresponding weight_scale_inv tensor of shape (ceil(rows/128), ceil(cols/128)).

python

import torch
from safetensors.torch import load_file, save_file

mtp = load_file("Qwen3.6-27B-FP8/mtp.safetensors")

weights = {k: v for k, v in mtp.items() if not k.endswith("weight_scale_inv")}
scales = {k: v for k, v in mtp.items() if k.endswith("weight_scale_inv")}

dequantized = {}
for k, v in weights.items():
    scale_key = k + "_scale_inv"
    if scale_key in scales:
        scale = scales[scale_key].to(torch.float32)
        w = v.to(torch.float32)
        bs = 128
        scale_expanded = scale.repeat_interleave(bs, dim=0).repeat_interleave(bs, dim=1)
        scale_expanded = scale_expanded[:w.shape[0], :w.shape[1]]
        dequantized[k] = (w * scale_expanded).to(torch.bfloat16)
    else:
        dequantized[k] = v.to(torch.bfloat16) if v.is_floating_point() else v

save_file(dequantized, "Qwopus3.6-27B-Abliterated/mtp.safetensors")

2. Patch the safetensors index

Add MTP weight map entries to model.safetensors.index.json, pointing at mtp.safetensors. Remove any weight_scale_inv entries.

python

import json

with open("Qwopus3.6-27B-Abliterated/model.safetensors.index.json") as f:
    idx = json.load(f)

fp8_idx = json.load(open("Qwen3.6-27B-FP8/model.safetensors.index.json"))
mtp_entries = {k: "mtp.safetensors" for k, v in fp8_idx["weight_map"].items()
               if k.startswith("mtp.") and "scale_inv" not in k}
idx["weight_map"].update(mtp_entries)
idx["weight_map"] = {k: v for k, v in idx["weight_map"].items() if "scale_inv" not in k}

with open("Qwopus3.6-27B-Abliterated/model.safetensors.index.json", "w") as f:
    json.dump(idx, f, indent=2)

3. Convert and quantize

bash

python3 convert_hf_to_gguf.py Qwopus3.6-27B-Abliterated/ \
  --outfile Qwopus3.6-27B-Abliterated-MTP-BF16.gguf --outtype bf16

llama-quantize Qwopus3.6-27B-Abliterated-MTP-BF16.gguf \
  Qwopus3.6-27B-Abliterated-MTP-Q8_0.gguf Q8_0

llama-quantize Qwopus3.6-27B-Abliterated-MTP-BF16.gguf \
  Qwopus3.6-27B-Abliterated-MTP-Q4_K_M.gguf Q4_K_M

Acceptance rate note

The grafted MTP head was trained against vanilla Qwen3.6-27B hidden states. Qwopus's SFT (~12K Claude reasoning samples) and croll83's abliteration shift the activation space, reducing draft acceptance from ~76% (on vanilla-derived heretic-v2) to ~55% average. This is consistent with kasimat's findings on AEON (~58% acceptance with a similar graft). The net throughput gain is still significant: 80–99 tok/s vs ~42 tok/s without MTP.

Retraining the MTP head against Qwopus's actual hidden states would likely recover acceptance to 70%+. This is left as future work.

Credits

Jackrong — Qwopus series, Claude 4.6 Opus reasoning distillation, complete fine-tuning guide
croll83 — Abliteration of Qwopus3.6, Blackwell-optimized llama.cpp fork
kasimat — Pioneered the MTP graft technique on AEON
sujitvasanth — Root cause analysis of the ISWA get_can_shift() bug
Qwen Team — Qwen3.6-27B base model and MTP architecture
HuiHui-AI — Abliteration methodology inspiration

License

Apache 2.0 (inherited from Qwen3.6-27B base model).

About

Built by William Pieh at PiehSoft LLC. This model was created during a deep benchmarking and model engineering session, documented in full. If you find it useful, a star would be appreciated.

PiehSoft/Qwopus3.6-27B-Abliterated-MTP-GGUF

Qwopus3.6-27B-Abliterated-MTP-GGUF

What's in this model

⚠️ Abliteration caveat: partial refusal behavior

Available quantizations

Benchmark results (RTX Pro 6000 Blackwell, 96GB VRAM)

Q8_0 results

Baseline comparison (same hardware, Qwen3.6-27B heretic-v2 MTP GGUF)

Recommended serve commands

Best config (MTP + prompt caching)

For 24GB GPUs (Q4_K_M)

⚠️ Correction: --cache-ram removed from recommended configs

Prompt caching on hybrid GDN models — findings and workarounds

Root cause: two bugs in llama.cpp (as of build bb28c1fe2)

The --no-context-shift workaround

Frontend-specific behavior (tested with --no-context-shift enabled)

vLLM findings (updated May 2026)

What does NOT work

Upstream issues

MTP graft methodology

1. Dequantize the FP8 MTP shard

2. Patch the safetensors index

3. Convert and quantize

Acceptance rate note

Credits

License

About

PiehSoft/
Qwopus3.6-27B-Abliterated-MTP-GGUF

⚠️ Correction: `--cache-ram` removed from recommended configs

The `--no-context-shift` workaround

Frontend-specific behavior (tested with `--no-context-shift` enabled)