Gemma 4 12B (the "Unified" release, June 2026) is an encoder-free unified multimodal model: text, image, audio, and video all project straight into a single decoder-only transformer, with a context window of up to 256K tokens. Abliteration touches only the language weights, so those capabilities carry over unchanged.
Not sure which to pick? Start with Q4_K_M. Go up to Q5/Q6/Q8 if you have the memory and want maximum fidelity; drop to Q3/Q2 only if you're memory-constrained.
Multimodal projector (for image/audio input — see Multimodal):
llama.cpp (auto-downloads the chosen quant from this repo):
bash
1# Interactive chat2llama-cli -hf zaakirio/gemma-4-12b-it-uncensored-GGUF:Q4_K_M --jinja
34# OpenAI-compatible server with web UI5llama-server -hf zaakirio/gemma-4-12b-it-uncensored-GGUF:Q4_K_M --jinja -c 4096
-c sets the context window. Gemma 4 12B supports up to 256K tokens (-c 262144); raise it for long chats/documents as memory allows — 4096 is just a light default.
Or with a file you've already downloaded:
llama-cli -m gemma-4-12b-it-uncensored-Q4_K_M.gguf --jinja -p "Hello, who are you?"
The chat template is embedded in the GGUF — chat-aware tools apply it automatically (always pass --jinja with llama.cpp). For reference, Gemma 4 introduced new turn tokens — <|turn> / <turn|>, replacing Gemma 3's <start_of_turn> / <end_of_turn> — and now supports a dedicated system role:
Thinking mode: Gemma 4 has a reasoning channel. To disable it, pass --chat-template-kwargs '{"enable_thinking":false}' to llama-server.
Multimodal
In llama.cpp, multimodal input ships via a separate projector file; the language .gguf alone is text-only and will reject media. This repo includes mmproj-gemma-4-12B-it-bf16.gguf, which carries both a vision and an audio encoder (clip.has_vision_encoder and clip.has_audio_encoder).
In practice:image and audio input both work. Verified with llama-mtmd-cli: it accepts .wav/.mp3 directly and produced an accurate multilingual transcription of real speech. llama.cpp flags audio as experimental ("audio input is in experimental stage and may have reduced quality"), so treat ASR quality as best-effort. Video is handled as sampled frames (plus audio) and depends on your client doing the frame extraction.
When you load via -hf, llama.cpp auto-downloads the projector from this repo — images just work:
One-shot image or audio from the CLI with llama-mtmd-cli:
bash
1# Image2llama-mtmd-cli -m gemma-4-12b-it-uncensored-Q4_K_M.gguf \3 --mmproj mmproj-gemma-4-12B-it-bf16.gguf --jinja \4 --image photo.png -p "What's in this image?"56# Audio (experimental in llama.cpp; 16 kHz WAV)7llama-mtmd-cli -m gemma-4-12b-it-uncensored-Q4_K_M.gguf \8 --mmproj mmproj-gemma-4-12B-it-bf16.gguf --jinja \9 --audio clip.wav -p "Transcribe this audio."
The projector is the unmodified Gemma 4 one (abliteration leaves it untouched). bf16 is preferred — it's small enough that quantizing it buys nothing.
Faster generation with MTP (speculative decoding)
Gemma 4 ships a small (~0.4 B) multi-token-prediction (MTP) drafter (google/gemma-4-12B-it-assistant) that speeds up generation through self-speculative decoding: the drafter proposes several tokens at once and the full model verifies them in a single forward pass. Output is identical to normal decoding — speculative decoding is mathematically exact, so it changes speed only, never quality, and the model stays exactly as uncensored: the main model verifies and decides every emitted token; the drafter merely proposes. (Google state the same — MTP "guarantee[s] the exact same quality as standard autoregressive generation" — and the implementation PR confirmed it by reproducing Gemma's published AIME-26 score.)
For that same reason the drafter does not need to be abliterated: this repo ships Google's unmodified upstream Gemma 4 drafter (the file is renamed mtp-gemma-4-12b-it-uncensored.gguf only so it sits beside the trunk — the weights are upstream), and your decensored trunk verifies every token. It's alignment-independent, exactly like the multimodal projector above.
MTP for Gemma 4 landed in llama.cpp PR #23398 (merged 2026-06-07; see also support discussion #22735), so it needs a llama.cpp build from that date or later.
bash
1# Grab the trunk + the MTP drafter2hf download zaakirio/gemma-4-12b-it-uncensored-GGUF \3 --include "gemma-4-12b-it-uncensored-Q4_K_M.gguf""mtp-gemma-4-12b-it-uncensored.gguf" --local-dir ./
45# Serve with MTP enabled6llama-server -m gemma-4-12b-it-uncensored-Q4_K_M.gguf \7 --model-draft mtp-gemma-4-12b-it-uncensored.gguf \8 --spec-type draft-mtp --spec-draft-n-max 4\9 -ngl 99 --flash-attn auto --jinja
⚠️ MTP helps on GPUs but is slower on Apple Silicon
The benefit depends entirely on the backend.
On a GPU it's a win. The implementation PR measured >2× throughput on the dense Gemma 4 (31B) model — NVIDIA DGX Spark, ~0.59 mean draft acceptance at --spec-draft-n-max 4 — and Unsloth report 1.5–2.2× for Gemma 4 with MTP. (PR #23398, Unsloth MTP guide.)
On Apple Silicon (Metal) it's a regression. Measured here on this model — Q4_K_M, Apple M4 Pro, Gemma's default sampling (--temp 1.0 --top-p 0.95 --top-k 64), through llama-server:
M4 Pro / Metal · Q4_K_M
Generation
Draft acceptance
MTP off (baseline)
~27 tok/s
—
MTP on (--spec-draft-n-max 4)
~12 tok/s (≈2× slower)
~0.45
The drafts are accepted (~45%), but on Metal the draft-evaluation overhead outweighs the decode steps it saves. This isn't specific to this model: an independent report (#23752 — Qwen3.5-9B on an M1 Max) found MTP slower at every setting, even at 100% acceptance, so no sampling tweak rescues it on Metal.
Recommendation: enable MTP when serving on an NVIDIA (CUDA) GPU; leave it off on a Mac. On a GPU, greedy / low-temperature sampling raises acceptance and widens the speedup.
About the base model
A decensored derivative produced with Heretic (automatic directional ablation). Compared with the original:
Metric
Decensored
Original
Refusals (/100 harmful prompts)
23
99
KL divergence (harmless prompts)
0.043
0 (by definition)
The refusal count is Heretic's keyword heuristic, which is known to over-count (it flags disclaimer-wrapped compliance as a refusal; ~11% precision per arXiv:2512.13655). We report only the measured marker figure and did not run a classifier-based eval on this model, so real compliance is likely higher. See the source model card for parameters and details.
Intended use & disclaimer
This model has had its refusal behaviour substantially removed and will comply with requests the original would have declined. Provided for research and unrestricted local use. You are responsible for how you use it and for complying with applicable law and the base model's Gemma license, which carries over to this derivative. Not for all audiences.
MTP drafter (mtp-gemma-4-12b-it-uncensored.gguf): the unmodified upstream google/gemma-4-12B-it-assistant (~0.4 B), Q8_0 GGUF via unsloth/gemma-4-12b-it-GGUF. Not abliterated — it only proposes tokens; the decensored trunk verifies every one, so output is identical to running this model alone.
