This repository includes a quantized language-model GGUF and a separate multimodal projector GGUF. Use both files together to preserve image-conditioned generation.

Files

File	Role	Notes
`Gemma4-12B-Uncensored-Q4_K_M.gguf`	Main language model	Q4_K_M quantization, practical default for local inference
`mmproj-Gemma4-12B-Uncensored-BF16.gguf`	Multimodal projector	Required for image/audio-conditioned Gemma 4 unified inference

Usage

Recommended with a recent llama.cpp build:

bash

llama-mtmd-cli \
  -m Gemma4-12B-Uncensored-Q4_K_M.gguf \
  --mmproj mmproj-Gemma4-12B-Uncensored-BF16.gguf \
  --image image.png \
  --jinja \
  -p "Describe the image." \
  -n 256

With Hugging Face auto-download:

llama-mtmd-cli -hf toandev/Gemma4-12B-Uncensored-GGUF:Q4_K_M --image image.png --jinja -p "Describe the image."

llama.cpp multimodal support is under active development. For Gemma 4, prefer current llama.cpp builds and keep --jinja enabled so the model-specific chat template is used correctly.

Smoke Test

Local conversion and smoke test used llama.cpp commit 18ef86e. The Q4_K_M model loaded successfully with the BF16 mmproj, processed a red-rectangle image, and identified the red visual content during generation.

Notes

This model is intended for research and controlled evaluation. Users are responsible for complying with the Gemma license, applicable platform policies, and local regulations.

Copyright

This model is a derivative of google/gemma-4-12B-it and remains subject to the upstream Gemma license and terms.