Q4_0-q4emb is an experimental quantization where token_embd.weight is kept in Q4_0 instead of Q6_K precision quantization typically used by llama.cpp.

This follows a similar approach to recent QAT experiments for Gemma models, where preserving the original Q4-trained embedding format may better match the intended QAT behavior.

Initial testing showed similar draft acceptance rates to the default Q4_0 quant, with a small speed advantage, though more benchmarking is needed.

Example

bash

llama-server \
  -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf \
  -md gemma-4-26b-A4B-it-assistant-Q4_0.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

Recommended values:

--spec-draft-n-max 2 for general use
--spec-draft-n-max 3 for coding workloads

RachidAR/gemma-4-26B-A4B-it-qat-assistant-q4_0-gguf

Gemma 4 26B A4B Assistant GGUF

Update

Recommendations

Files

Q4 Embedding Variant

Example

RachidAR/
gemma-4-26B-A4B-it-qat-assistant-q4_0-gguf