Q4_0 GGUF quantization of Qwen3.6-35B-A3B (Mixture-of-Experts, ~3B active parameters per token) that retains the Multi-Token Prediction (MTP) tensors, so it can self-speculate with llama.cpp's --spec-type draft-mtp — no separate draft model required.


Architecture	Qwen3.6 MoE (A3B — ~3B active params/token)
Quantization	`Q4_0`
File	`Qwen3.6-35B-A3B-Q4_0.gguf` (~19 GB, single file)
Speculative decoding	MTP (built in)

Requirements

MTP is not yet in mainline llama.cpp. Build the branch that carries the draft-mtp speculative type:

bash

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout mtp-exp     # branch with --spec-type draft-mtp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

Run

bash

llama-server \
  -m Qwen3.6-35B-A3B-Q4_0.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  -c 262144 -ngl 99

Download the weights:

bash

pip install -U "huggingface_hub[hf_transfer]"
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download shujunyi/Qwen3.6-35B-A3B-MTP-Q4_0 \
  Qwen3.6-35B-A3B-Q4_0.gguf --local-dir .

Notes

On a single A100-40GB this quant runs fully offloaded; observed ~145 tok/s with MTP speculative decoding at ~63% draft-token acceptance.
Q4_0 is a simple/legacy quant chosen for a clean quality baseline; for the best size/quality tradeoff consider an IQ4_XS / imatrix build instead.