Weights are byte-identical to the upstream NVIDIA release. This mirror exists to provide a pinned, stable reference for the gemma4 deployment project running on DGX Spark (GB10). Use the upstream repo if you don't need a pinned copy.

Credits

Base model: google/gemma-4-31B-it by Google DeepMind
NVFP4 quantization: NVIDIA via ModelOpt
License: Apache-2.0 (Gemma license)

Verified performance

Measured on DGX Spark (GB10 Blackwell, SM12.1, 128 GB unified LPDDR5X) with vLLM 0.19.1, Marlin NVFP4 backend, BF16 KV cache:

Test	Prompt	Output	Time	Decode tok/s
EN — short Q&A	25	8	1.3s	6.1
EN — chain-of-thought	40	126	18.3s	6.9
PL — short explanation	30	116	16.9s	6.9
Python code generation	36	400	57.8s	6.9

Steady-state decode: ~6.9 tok/s (memory-bandwidth bound; Marlin runs NVFP4 as W4A16 GEMM since SM12.1 has no native CUTLASS FP4 kernel).

Usage

bash

docker run --rm --runtime=nvidia --gpus all \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_NVFP4_GEMM_BACKEND=marlin \
  vllm/vllm-openai:gemma4-cu130 \
  gdubicki/Gemma-4-31B-IT-NVFP4 \
  --quantization modelopt_fp4 \
  --dtype bfloat16 \
  --gpu-memory-utilization 0.40 \
  --max-model-len 32768

Full deployment scripts: https://github.com/grzegorzdubicki-ai/gemma4