Requires a llama.cpp build from 2026-06-04 or later (PR #24118, which added Gemma4Unified conversion + inference support). Earlier builds will fail to load these files.

Files

File	Quant	Size
`gemma-4-12B-it-uncensored-Q4_K_M.gguf`	Q4_K_M	7.4 GB
`gemma-4-12B-it-uncensored-Q8_0.gguf`	Q8_0	12.7 GB

Usage

bash

# From HuggingFace (auto-downloads)
llama-server -hf TrevorJS/gemma-4-12B-it-uncensored-GGUF -c 8192

# From local file
llama-server -m gemma-4-12B-it-uncensored-Q4_K_M.gguf -c 8192

Then open http://localhost:8080 for the chat UI.

Details

These are GGUF quantizations of TrevorJS/gemma-4-12B-it-uncensored, an abliterated (uncensored) version of google/gemma-4-12B-it. Refusal behavior has been removed using norm-preserving biprojected abliteration.

See the bf16 model card for full method details, before/after refusal rates, and cross-dataset validation results.

Source code: TrevorJS/gemma-4-abliteration

Ishowbackup/gemma-4-12B-it-uncensored-GGUF

gemma-4-12B-it-uncensored (GGUF)

Files

Usage

Details

Ishowbackup/
gemma-4-12B-it-uncensored-GGUF