Turbo quant (WHT-rotated) GGUF quantizations of Qwen/Qwen3.5-9B-Instruct,
quantized with imatrix calibration using a curated Stack Overflow / Software Engineering SE dataset.

Intended Use: Speculative Decoding Draft Model for Qwen3.6-27B

These quants are specifically designed as draft models for speculative decoding with Qwen3.6-27B.

Qwen3.5-9B and Qwen3.6-27B share the same tokenizer (n_vocab=248320) and architecture family (qwen35), making them compatible for speculative decoding out of the box. Early
testing with Qwen3.5-4B Q6_K as draft showed ~67% per-token acceptance rate against
Qwen3.6-27B — the 9B base should push this higher.

Quant Details

File	Format	Size	bpw
Qwen3.5-9B-TQ2_0.gguf	TQ2_0 (ternarization)	~2.3GB	2.06
Qwen3.5-9B-TQ3_1S.gguf	TQ3_1S (WHT-rotated)	~4.5GB	4.00

Quantized using a custom llama.cpp fork with
optimized ROCm/HIP TQ kernel support, imatrix-calibrated on Stack Overflow and
Software Engineering SE Q&A data.

Usage with llama.cpp

bash

llama-server \                                                                                                                           
  --model Qwen3.6-27B-Q6_K.gguf \
  --model-draft Qwen3.5-9B-TQ3_1S.gguf \
  -ngl 999 -ngld 999 \                                                                                                                     
  --draft-max 12 --draft-p-min 0.75 \                                                                                                      
  --parallel 1                                                                                                                             
                                                                                                                                           
Hardware Tested                                                                                                                            
                                                                                                                                           
- AMD Radeon AI PRO R9700 (32GB, gfx1201 RDNA4) via ROCm                                                                                   
                                                                                                                                           
Calibration Data                                                                                                                           
                                                                                                                                         
Imatrix calibrated on ~1000 examples sampled from Stack Overflow and Software
Engineering Stack Exchange Q&A pairs, formatted with the Qwen chat template.
Domain-matched to coding and software engineering tasks.

mad-lab-ai/Qwen3.5-9b-tq-gguf

Qwen3.5-9B-Instruct TQ2_0 & TQ3_1S (Turbo Quant GGUF)

Intended Use: Speculative Decoding Draft Model for Qwen3.6-27B

Quant Details

Usage with llama.cpp

mad-lab-ai/
Qwen3.5-9b-tq-gguf