Q8_0 GGUF quantization of the z-lab/gemma-4-26B-A4B-it-DFlash draft model, produced for the Lucebox dflash engine (speculative decoding for google/gemma-4-26B-A4B-it).

Source: z-lab/gemma-4-26B-A4B-it-DFlash (BF16 safetensors)
Tool: quantize_gemma_dflash_q8.py (parameterized variant of dflash/scripts/quantize_draft_q8.py)
Size: 0.46 GB (53% of BF16)
Arch: gemma4-dflash-draft
Layers: 5
Hidden: 2816
n_head / n_head_kv: 32 / 8
head_dim: 128
intermediate_size: 5632
vocab_size: 262144
rope_theta: 1e6, rms_eps: 1e-6
sliding_window: 2048, final_logit_softcapping: 30.0
DFlash: n_target_layers=30, target_layer_ids=[1,6,11,17,22,27], block_size=16, mask_token_id=4
Tensors: projection weights → Q8_0, norm weights → F32 (precision-critical, tiny)

Pairs with

Target: google/gemma-4-26B-A4B-it (run as Q4_K_M GGUF via unsloth/gemma-4-26B-A4B-it-GGUF or equivalent)

Notes

The dflash-specific singletons dflash.fc.weight and dflash.hidden_norm.weight bridge target hidden states into the draft. Do not re-quantize with stock llama-quantize — it strips these tensors. Use the script above.
Loader support for gemma4-dflash-draft in lucebox-hub is the next step after PR #232 (gemma4 target adapter).

Usage

bash

# With dflash_server (once gemma4-dflash-draft arch is wired in the loader)
dflash_server gemma-4-26B-A4B-it-UD-Q4_K_M.gguf --draft gemma-4-26B-A4B-it-DFlash-q8_0.gguf

Conversion command

bash

PYTHONPATH=lucebox-hub/dflash/deps/llama.cpp/gguf-py \
python3 quantize_gemma_dflash_q8.py \
  gemma-4-26B-A4B-it-DFlash/ \
  gemma-4-26B-A4B-it-DFlash-q8_0.gguf \
  --name gemma-4-26B-A4B-it-DFlash-Q8_0

Lucebox/gemma-4-26B-A4B-it-DFlash-GGUF

gemma-4-26B-A4B-it DFlash Draft — Q8_0 GGUF

Pairs with

Notes

Usage

Conversion command

Lucebox/
gemma-4-26B-A4B-it-DFlash-GGUF