The Q6_K file is about 37 MiB larger than the Q4_K_S file in this local build. Try Q4_K_S first if you are tight on VRAM; try Q6_K if you want the slightly heavier assistant head.

File

File	Quant	Size	SHA256
`gemma-4-26B-A4B-it-assistant-Q6_K.gguf`	Q6_K	344 MiB	`49e59e9e0e097f97751d08b9bd00edf3bd2316b188bd129bd5098eccf8378d95`

How this was made

Source file:

gemma-4-26B-A4B-it-assistant-F16.gguf
Source repository: Radamanthys11/Gemma-4-26B-A4B-it-assistant-GGUF
Upstream base model: google/gemma-4-26B-A4B-it-assistant

Quantized with ik_llama.cpp:

Commit: 397150ca MTP: faster recurrent state restore (#1791)
llama-quantize SHA256: 15ca21cbd3881326cec391325d663fed94122fe87a42d84373682b1612d9dc52
No imatrix was used.
This was quantized from F16, not requantized from Q8.

Command:

bash

/home/joel/Projects/ik_llama.cpp/build-cuda/bin/llama-quantize \
  gemma-4-26B-A4B-it-assistant-F16.gguf \
  gemma-4-26B-A4B-it-assistant-Q6_K.gguf \
  Q6_K \
  12

Quantizer output reported:

text

main: build = 4482 (5fe010f0)
llama_model_quantize_internal: model size  =   800.57 MB
llama_model_quantize_internal: quant size  =   328.40 MB

Usage

Use this as the draft model alongside a Gemma 4 26B-A4B target GGUF:

bash

./build/bin/llama-server \
  -m gemma-4-26B-A4B-it-UD-IQ4_XS.gguf \
  -c 20480 \
  --model-draft gemma-4-26B-A4B-it-assistant-Q6_K.gguf \
  --spec-type mtp \
  -ngld 999 \
  --draft-max 2 \
  --draft-p-min 0.0 \
  -ctk q4_0 \
  -ctv q4_0 \
  --jinja

--draft-max 1 and --draft-max 2 are both reasonable starting points. Local results are workload-sensitive, so treat this as a practical assistant option rather than a universal speed claim.

Local smoke test

Smoke-tested locally with:

Target: gemma-4-26B-A4B-it-UD-IQ4_XS.gguf
Draft: this Q6_K assistant
ik_llama.cpp commit: 397150ca
Context: 20480
--draft-max 2

On a short local chat-completions smoke against the Q4_K_S native assistant, the Q6_K head was larger but steadier:

Draft	Size	Mean TG	Acceptance
Q4_K_S	307 MiB	35.62 tok/s	220/360 = 61.1%
Q6_K	344 MiB	45.71 tok/s	232/360 = 64.4%

This is only a tiny local check on one RTX 4070 setup, but it suggests Q6_K may be a useful middle point between the smaller Q4_K_S head and the larger public Q8_0 head.

License

This quantization follows the Gemma license inherited from the upstream Google Gemma model.

ji-farthing/gemma-4-26B-A4B-it-assistant-Q6_K-GGUF

Gemma 4 26B-A4B IT Assistant Q6_K GGUF

Related quant

File

How this was made

Usage

Local smoke test

License

ji-farthing/
gemma-4-26B-A4B-it-assistant-Q6_K-GGUF