This GGUF was quantized from the BF16 safetensors using llama.cpp (build 537). The source weights were converted to F16 GGUF, then quantized to NVFP4 format.

NVFP4 (NVIDIA FP4) uses 4-bit floating point quantization optimized for NVIDIA Blackwell (B-series) GPUs.

Files

File	Size	Description
`diffusiongemma-26b-a4b-it-nvfp4.gguf`	~13.4 GB	NVFP4 quantized model weights
`mmproj-diffusiongemma-26b-a4b-f16.gguf`	~1.19 GB	Vision encoder (F16)

Usage

llama.cpp

bash

llama-server \
  -m diffusiongemma-26b-a4b-it-nvfp4.gguf \
  --mmproj mmproj-diffusiongemma-26b-a4b-f16.gguf \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

Hardware Requirements

Minimum: 16 GB VRAM for partial offload
Recommended: 24+ GB VRAM for full GPU offload

License

Apache 2.0 - same as the base model.

Note: Vision support requires the mmproj file. The mmproj is shared between the NVFP4 and MXFP4 variants.

FreedomAISVR/DiffusionGemma-26B-A4B-it-NVFP4-GGUF

DiffusionGemma 26B-A4B-it - NVFP4 GGUF

About the Model

Quantization

Files

Usage

llama.cpp

Hardware Requirements

License

FreedomAISVR/
DiffusionGemma-26B-A4B-it-NVFP4-GGUF