Qwen3.6-27B-uncensored-abliterated-MTP-i1-IQ4_XS-GGUF(Smaller)
♥ MTP Inference-Accelerated Model Optimized for 16GB VRAM GPUs ♥
This model is a
native MTP (Multi-Token Prediction) capable version, extracted from the Dense backbone of
llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-GGUF and quantized. It supports longer contexts, features
uncensored (abliterated) characteristics, and significantly boosts per-token inference speed.
For use cases requiring longer contexts (e.g., 128K+) at approximately 20 tokens/s inference speed, consider this model:
https://huggingface.co/lemonyins/Qwen3.6-27B-uncensored-abliterated-i1-IQ4_XS-GGUF-Smaller
Key Highlights
- MTP Speculative Decoding: Native Multi-Token Prediction draft generation boosts inference from 20 → 35 tokens/s (75% improvement)
- High Speed at Long Contexts: 20 tokens/s at 50K context — 2× faster than non-MTP models (only 10 tokens/s)
- 70% Draft Acceptance Rate: spec-draft-n-max=2 is optimal; higher values do not improve acceptance
- 16GB VRAM, up to 60K context: Fully fits on a single GPU with TurboQuant KV Cache (turbo4)
- FFN Layer IQ3_S Mixed Precision: Further reduces model size, freeing VRAM for KV Cache
- Uncensored Model: Abliterated to remove content restrictions, suitable for deep research
Innovation
This model inherits the mixed-precision quantization strategy from
Qwen3.6-27B-uncensored-abliterated-i1-IQ4_XS-GGUF-Smaller:
attn_qkv /
attn_k /
attn_v /
attn_output /
output layers remain at
IQ4_XS, while
ffn_down /
ffn_up /
ffn_gate layers are downgraded to
IQ3_S. On top of this,
the core breakthrough is MTP support — the base model preserves the native MTP Head, enabling parallel generation of multiple draft tokens during inference, which are accepted in one batch after verification by the target model, significantly reducing the number of serial decoding steps.
MTP Inference Performance
Tested on: NVIDIA RTX 4060 Ti 16GB, llama.cpp (turboquant + mtp branch)
| Scenario | Speed |
|---|
| Short context (non-MTP model) | 19 tokens/s |
| Short context (MTP model) | 35 tokens/s |
| Long context 50K (non-MTP model) | 10 tokens/s |
| Long context 50K (MTP model) | 20 tokens/s |
| Draft acceptance rate | 70% |
Memory Usage (TurboQuant KV Cache)
| Version | Context Length | KV Cache | VRAM Usage |
|---|
IQ4_XS-FFN-IQ3_S (this model) | 60K | kv=turbo4 | ~15.4 GB |
IQ4_XS-FFN-IQ3_S (this model) | 48K | kv=turbo4 | ~15.2 GB |
IQ4_XS-FFN-IQ3_S (this model) | 32K | k=q8_0,v=turbo4 | ~15.3 GB |
- Note: After testing, setting the context to 48K will be more stable and less likely to cause out-of-memory errors.
- Note: llama.cpp automatically upgrades
cache-type-k to q8_0, which limits context to ~32K on the same VRAM budget. See the Run Command section for the solution.
KV Cache Precision Comparison (Turbo4 vs q8_0)
By setting TURBO_AUTO_ASYMMETRIC=0, the KV Cache uses the turbo4 format instead of the auto-upgraded q8_0, providing significant VRAM savings with minimal perplexity impact:
English novel test:
| KV Cache Config | Perplexity | Difference |
|---|
| k=q8_0 + v=turbo4 | 1.3436 +/- 0.00539 | Baseline |
| kv=turbo4 | 1.3536 +/- 0.00551 | +0.74% only |
Code test:
| KV Cache Config | Perplexity | Difference |
|---|
| k=q8_0 + v=turbo4 | 1.2312 +/- 0.00157 | Baseline |
| kv=turbo4 | 1.2322 +/- 0.00157 | +0.08% only |
Conclusion: kv=turbo4 delivers significant VRAM savings with minimal perplexity loss (0.1%–0.7%), making 60K context feasible.
Methodology
- Base model: llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-GGUF — an uncensored GGUF with native MTP Head preserved
- Extraction and quantization: Dense backbone extracted (27B), quantized using TurboQuant technology stack with mixed precision
- Quantization types:
attn_qkv, attn_k, attn_v, attn_output, output: IQ4_XSffn_down, ffn_up, ffn_gate: IQ3_S- Other layers: default
IQ4_XS
Run Command
16GB VRAM | 60K Context | MTP Acceleration
1set TURBO_AUTO_ASYMMETRIC=0
2
3llama-server.exe ^
4 -m Qwen3.6-27B-uncensored-abliterated-MTP-i1-IQ4_XS-FFN-IQ3.gguf ^
5 --parallel 1 ^
6 --spec-type mtp ^
7 --spec-draft-n-max 2 ^
8 -c 61440 ^
9 -ngl 999 ^
10 --flash-attn on ^
11 -ctk turbo4 ^
12 -ctv turbo4 ^
13 --host 0.0.0.0 ^
14 --port 1234
Key Parameter Descriptions
| Parameter | Description |
|---|
--spec-type mtp | Enable MTP speculative decoding mode |
--spec-draft-n-max 2 | Max draft tokens; 2 is optimal (higher values do not improve acceptance rate in testing) |
-ctk turbo4 / -ctv turbo4 | Use turbo4 format for Key/Value Cache; requires TURBO_AUTO_ASYMMETRIC=0 to take effect |
set TURBO_AUTO_ASYMMETRIC=0 | Prevents automatic K Cache upgrade to q8_0, ensuring turbo4 is used and saving VRAM |
--flash-attn on | Enable Flash Attention for speedup |
-c 61440 | 60K context window |
About spec-draft-n-max
Extensive testing shows that --spec-draft-n-max 2 is the optimal configuration. The draft acceptance rate saturates at ~70%; increasing the draft count to 3 or higher does not improve actual output speed and only adds computational overhead.
Runtime Requirements
You need a llama.cpp fork that supports both TurboQuant and MTP:
This build fixes the TURBO_AUTO_ASYMMETRIC logic and works out of the box — no need to manually set the environment variable.
Caveats
- MTP is essential for speedup: You must use an MTP-capable llama.cpp fork and specify
--spec-type mtp, otherwise the MTP Head will not be activated
- TurboQuant is mandatory: Without TurboQuant KV Cache, 16GB VRAM cannot support 60K context
- Environment variable required: If using a non-lemonyins build, you must
set TURBO_AUTO_ASYMMETRIC=0 first; otherwise K Cache will be auto-upgraded to q8_0 and VRAM will be insufficient for 60K
- Vision module removed: There is insufficient VRAM to load the vision module, so this model is for text-only inference acceleration. For vision support, use: https://huggingface.co/lemonyins/Qwen3.6-27B-uncensored-abliterated-i1-IQ4_XS-GGUF-Smaller
Acknowledgments
- llmfan46 — Providing the native MTP-preserved uncensored base GGUF
- QuinsZouls — Providing the llama.cpp branch supporting both TurboQuant and MTP (llama-cpp-turboquant/llama-next)
- lemonyins — Providing precompiled binaries and fixing the K Cache auto-upgrade issue
- llama.cpp — The GGML / llama.cpp team and community
