Quant	Bits	Size	Notes
Q4_K_M	4	226 MB	Average quality
Q5_K	5	315 MB	High quality
Q6_K	6	367 MB	Very high quality
Q8_0	8	471 MB	Highest quality, near lossless, Recommended
BF16	16	874 MB	Full precision, reference file

Usage

Use in conjunction with existing Gemma 4 Quants, example config if using llama-server:

bash

[Gemma-4-26B-A4B-it-DFlash]
sm = layer
model = /mnt/gguf/Gemma-4-26B-A4B-it/Gemma-4-26B-A4B-it-Q8_0.gguf
model-draft = /mnt/gguf/Gemma-4-26B-A4B-it-DFlash/Gemma-4-26B-A4B-it-DFlash-Q8_0.gguf
spec-type = draft-dflash
spec-draft-n-max = 6 

(Note: For some reason I cannot get sm = tensor to work, it crashes on launch, pretty sure this is an issue in llama.cpp)

Original model

See the original model card for details on capabilities, benchmarks, and license.

Alittlehammmer/gemma-4-26B-A4B-it-DFlash-GGUF-llama.cpp

Gemma-4-26B-A4B-it-DFlash

Available quants

Usage

Original model

Alittlehammmer/
gemma-4-26B-A4B-it-DFlash-GGUF-llama.cpp