GGUF quantizations of igorls/gemma-4-12B-it-qat-q4_0-unquantized-heretic, a decensored ("abliterated") version of Google's Gemma 4 12B QAT-Q4_0 unquantized checkpoint, produced with Heretic v1.3.0.

Quant	Size	Notes
`Q4_0`	6.5 GB	Recommended. QAT-matched: the base was quantization-aware-trained for the Q4_0 grid, so this 4-bit quant best preserves the original quality
`Q4_K_M`	6.9 GB	Standard K-quant 4-bit alternative
`Q8_0`	11.8 GB	Near-lossless

[!NOTE] This model derives from Google's QAT-Q4_0 checkpoint. Quantization-aware training was calibrated specifically for the Q4_0 grid, so Q4_0 is the quant that realizes the QAT advantage (≈bf16 quality at 4-bit). Q4_K_M is a normal K-quant of the same weights and does not specifically leverage QAT.

v1.1 — thinking-mode fix

Gemma 4 is a thinking model: its refusal decision forms inside the chain-of-thought. The first release was abliterated/evaluated with thinking disabled, so it still refused once thinking was on (the default in Ollama / llama.cpp). v1.1 decensors the model with thinking enabled — the way it's actually used.

Performance

Metric	This model (v1.1)	Original
KL divergence	0.32	0 (by definition)
Refusals, thinking on (adversarial harmful set)	~22%	~99%

KL is high relative to a typical abliteration — that is the cost of suppressing refusal through the reasoning trajectory. The model complies with the vast majority of requests in normal use; a small fraction of extreme prompts may still be refused, and a stronger v2 is in progress.

Run with llama.cpp

llama-cli -hf igorls/gemma-4-12B-it-qat-q4_0-unquantized-heretic-GGUF:Q4_0

Run with Ollama

ollama run hf.co/igorls/gemma-4-12B-it-qat-q4_0-unquantized-heretic-GGUF:Q4_0

Requires a llama.cpp / Ollama build with Gemma 4 (gemma4) support.

Disclaimer

This is an abliterated model: its built-in safety guardrails and refusal behavior have been deliberately removed. As a result it will attempt to answer essentially any prompt and can produce content that is offensive, inaccurate, explicit, or otherwise harmful — content the original Gemma 4 would have refused.

No safety alignment. Do not rely on it to refuse unsafe requests or to self-moderate. Apply your own filtering, guardrails, and human review before any production or user-facing use.
You are solely responsible for how you use this model and for complying with all applicable laws and with the base model's Gemma 4 license (Apache 2.0).
Intended for adults (18+), for research, evaluation, and lawful creative use where permitted.
Provided as-is, without warranty of any kind. The author accepts no liability for any output produced or any use of this model.