FP8 (compressed-tensors, FP8_DYNAMIC W8A8) quantisation of bottlecapai/ThinkingCap-Qwen3.6-27B.

This is our default FP8 build. It keeps the entire self-attention path in bf16 (the attnbf16 recipe) and ships a working MTP / self-speculative-decoding head.

lm_head, the MTP head, the vision tower, the whole linear_attn (Gated-DeltaNet / SSM) block, and — the distinctive choice here — the entire self_attn path, including the attention output gate that Qwen3.5/3.6 fuses into q_proj. Only the MLPs (~17 B params) are FP8. Quantisation error on the fused multiplicative gate is qualitatively worse than on an additive projection, and only ~1/4 of the layers are full-attention, so keeping attention bf16 costs ~1.5 GiB (~4.6 %) and measurably lowers KLD. See recipe.yaml.

Accuracy vs the bf16 parent: KLD ≈ 0.0132 nats, ΔPPL ≈ +0.96 % (kld_thinkingcap-fp8-attnbf16.json, ppl_thinkingcap-fp8-attnbf16.json).

The bf16 MTP head is declared in quantization_config.ignore so vLLM loads it correctly (a bf16 MTP head regrafted into a compressed-tensors quant is otherwise mis-loaded and yields 0 % draft acceptance).

--max-num-seqs 32 (or lower) is required — Qwen3.6 is a hybrid linear-attention model and the default max_num_seqs exceeds the available Mamba cache blocks.

Loads natively in vLLM / SGLang. Text + vision + MTP-presence smoke tests pass; the vision tower correctly describes images.