This follows a similar approach used by recent Unsloth Gemma QAT releases. Initial testing showed comparable draft acceptance rates while reducing model size.

Example

bash

llama-server \
  -m gemma-4-31B-it-qat-Q4_0.gguf \
  -md gemma-4-31b-qat-it-assistant-Q4_0-Q4emb.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

Recommended values:

--spec-draft-n-max 2 for general use
--spec-draft-n-max 3 for coding workloads
--spec-draft-n-max 4 may help on highly repetitive tasks

RachidAR/gemma-4-31B-it-qat-Q4_0-Q4emb-MTP-assistant-gguf

Gemma 4 31B QAT MTP Assistant GGUF

Recommendations

Files

Notes

Example

RachidAR/
gemma-4-31B-it-qat-Q4_0-Q4emb-MTP-assistant-gguf