This GGUF was quantized from the BF16 safetensors using llama.cpp (build 537). The source weights were converted to F16 GGUF, then quantized to MXFP4 format.

MXFP4 (Microscaling FP4) uses block-wise quantization with shared exponents per block.

Files

File	Size	Description
`diffusiongemma-26b-a4b-it-mxfp4.gguf`	~12.7 GB	MXFP4 quantized model weights
`mmproj-diffusiongemma-26b-a4b-f16.gguf`	~1.19 GB	Vision encoder (F16)

Usage

llama.cpp

bash

llama-server \
  -m diffusiongemma-26b-a4b-it-mxfp4.gguf \
  --mmproj mmproj-diffusiongemma-26b-a4b-f16.gguf \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

Hardware Requirements

Minimum: 16 GB VRAM for partial offload
Recommended: 24+ GB VRAM for full GPU offload

License

Apache 2.0 - same as the base model.

Note: Vision support requires the mmproj file. The mmproj is shared between the NVFP4 and MXFP4 variants.

FreedomAISVR/DiffusionGemma-26B-A4B-it-MXFP4-GGUF

DiffusionGemma 26B-A4B-it - MXFP4 GGUF

About the Model

Quantization

Files

Usage

llama.cpp

Hardware Requirements

License

FreedomAISVR/
DiffusionGemma-26B-A4B-it-MXFP4-GGUF