This is not a single uniform Q4 or Q3 file. It is a mixed-precision build where different tensor groups keep different quant types according to sensitivity and size budget.

Mix Formula

This published 13GB file follows a mixed recipe in this style:

token_embd -> q5_k
output -> q5_k
router -> q8_0
attn_q -> q6_k
attn_k -> q6_k
attn_v -> q6_k
attn_output -> q6_k
ffn_gate_up_exps -> mixed q4_k / q3_k
ffn_down_exps -> q4_0

Notes:

this file is MoE, so expert tensors are not laid out like the dense 31B recipe
ffn_gate_up_exps is the main mixed expert block
the 13GB release is therefore closer to a Q6_K + Q4_K/Q3_K expert mix than to a Q3-centered dense recipe

Importance Matrix (`imatrix`)

This release follows the same imatrix-guided quantization idea used in the 31B line.

Core formula:

I_j = Σ_t x_{t,j}^2

Where:

x_{t,j} is the activation value of channel j for token/sample step t
I_j is the accumulated importance score of that channel across calibration text

Practical meaning:

channels that activate more often and with larger magnitude get larger importance values
more important directions are better preserved during quantization
less important directions can be compressed more aggressively

imatrix does not use benchmark scores directly. It estimates sensitivity from activations collected on calibration data.

Multimodal Support

Yes. Multimodal remains supported when paired with:

mmproj-gemma-4-26b-a4b-it-f16.gguf

Notes:

the projector was preserved separately
the 13GB main file is text-side quantized in the same release style as the 31B line
image-text usage depends on loading mmproj together with the main model

Road

The working road was:

Keep the original HF Gemma 4 26B A4B it model as the source of truth.
Export the text model to F16 GGUF.
Preserve the multimodal projector as a separate file.
Build a mixed 13GB quantized release for local llama.cpp inference.
Publish the main GGUF together with the projector file.

Self Tests

Observed checks for the published 13GB release:

the GGUF file is valid and readable
the multimodal projector file is present in the repository
the release remains a multimodal package when used with mmproj

Note:

local experimental variants and local runtime behavior may differ from this published file
the README here describes the actual uploaded Hugging Face GGUF file, not a guessed local preset name

Environment Build

Minimal setup:

Install CUDA and a recent NVIDIA driver.
Build llama.cpp with CUDA support.
Keep the 13GB GGUF and mmproj together if you need vision.
Load both files together for multimodal inference.

Example server:

llama-server \
  -m 'google_gemma-4-26b-a4b-it-mix-13GB.gguf' \
  --mmproj 'mmproj-gemma-4-26b-a4b-it-f16.gguf' \
  -ngl 999 -fa on --ctx-size 4096 -np 1 --port 18081

Datasets And License Notes

This repository is a GGUF release of the Google base model.

License:

Apache-2.0
official license link: https://ai.google.dev/gemma/docs/gemma_4_license

Practical Summary

Use this version if you want:

a published mixed 13GB GGUF for Gemma 4 26B A4B it
multimodal support preserved through the separate mmproj
a MoE mixed quant release documented in recipe style instead of a generic quant summary

keyuan01/Gemma-4-26B-A4B-it-MixQ-13G-GGUF

Gemma 4 26B A4B it Mix-Quant 13GB GGUF

File

Size

What This Is

Quantization Type