You can deploy this model and start earning money today!

bombman/
gemma-4-12B-it-GGUF

Name: gemma-4-12B-it-GGUF
Availability: InStock

google/gemma-4-12B-it

Views

No views yet

Gemma 4 12B IT GGUF

GGUF quantizations of Google's Gemma 4 12B IT model.

Optimized for local inference with:

Ollama
LM Studio
Jan AI
llama.cpp
Open WebUI
Text Generation WebUI
KoboldCpp

Downloads

Quant	Recommended Use
Q4_K_S	Smallest size
Q4_K_M	⭐ Recommended
Q8_0	Maximum quality
F16	Further quantization

Quick Start

Ollama

Create a file named Modelfile

text

FROM ./gemma-4-12B-it-Q4_K_M.gguf

PARAMETER temperature 0.7
PARAMETER num_ctx 32768

SYSTEM You are a helpful AI assistant.

Create model:

ollama create gemma4-12b -f Modelfile

Run:

ollama run gemma4-12b

LM Studio

Download a GGUF file.
Open LM Studio.
Import the model.
Load the model.
Start chatting.

Recommended:

gemma-4-12B-it-Q4_K_M.gguf

Jan AI

Download a GGUF file.
Open Jan AI.
Models → Import Model
Select the GGUF file.
Start chatting.

Recommended:

gemma-4-12B-it-Q4_K_M.gguf

Open WebUI

Using Ollama:

ollama create gemma4-12b -f Modelfile

Then select:

gemma4-12b

inside Open WebUI.

llama.cpp

Run:

bash

./build/bin/llama-cli \
  -m gemma-4-12B-it-Q4_K_M.gguf \
  -cnv

Server:

bash

./build/bin/llama-server \
  -m gemma-4-12B-it-Q4_K_M.gguf

OpenAI-compatible endpoint:

http://localhost:8080/v1

Hardware Recommendations

Quant	RAM
Q4_K_S	10 GB+
Q4_K_M	12 GB+
Q8_0	18 GB+
F16	32 GB+

Which Version Should I Download?

Most Users

Q4_K_M

Best balance of:

Quality
Speed
Memory usage

Highest Quality

Q8_0

Smallest Size

Q4_K_S

Further Quantization

F16

Credits

Google for Gemma 4
ggml-org for llama.cpp
Ollama
LM Studio
Jan AI
Open WebUI

Disclaimer

This repository contains GGUF conversions only.

All original model weights and intellectual property belong to Google.

bombman/gemma-4-12B-it-GGUF