GGUF quantizations of
DuoNeural/Gemma4-12B-IT-Abliterated — an abliterated Gemma 4-12B-IT with the refusal direction surgically removed.
1# Download a quant
2huggingface-cli download DuoNeural/Gemma4-12B-IT-Abliterated-GGUF \
3 gemma4_12b_abliterated_Q4_K_M.gguf --local-dir ./
4
5# Run with llama.cpp
6./llama-cli -m gemma4_12b_abliterated_Q4_K_M.gguf \
7 -p "Write a haiku about hacking." \
8 -n 200 --temp 0.7
1from llama_cpp import Llama
2
3llm = Llama(
4 model_path="./gemma4_12b_abliterated_Q4_K_M.gguf",
5 n_ctx=4096,
6 n_gpu_layers=-1, # offload all layers to GPU
7)
8
9output = llm.create_chat_completion(
10 messages=[{"role": "user", "content": "Your prompt here"}],
11 max_tokens=512,
12 temperature=0.7,
13)
14print(output["choices"][0]["message"]["content"])
Congratulations to
OpenYourMind for being the first published abliteration of Gemma 4-12B-IT (Jun 3, 2026). Their approach uses diff-in-means on a labeled harmful/harmless set; ours uses orthogonal rank-1 projection via
heretic-llm. Two independent methods on the same base — a useful comparison point for the community. We are not affiliated and did not use their data.
