This repository hosts a complete, professional-grade suite of iMatrix (Importance Matrix) GGUF quantizations of culturerevolt/gemma-4-12b-heretic-abliterated.

The base model is an abliterated, fully decensored variant of Google's gemma-4-12b-it architecture, stripped of categorical refusal strings via norm-preserving directional ablation. This comprehensive GGUF suite is calibrated to preserve maximum linguistic entropy, narrative depth for creative fiction, and strict syntax parsing for agentic tools across multiple hardware profiles.

📊 Quick-Reference: Which Quant Should You Download?

All models (except the standard Q8_0 baseline) utilize a custom calculated importance matrix to prevent the loss of reasoning performance that standard static quants suffer from. Use this table to match the model files to your available hardware footprint:

File Name	Quant Type	Sizing / VRAM Target	Best Use Case
`gemma-4-12b-heretic-Q8_0.gguf`	Q8_0	24GB VRAM (RTX 3090 / 4090 / Mac Studio)	The Master Reference. Runs at a near-lossless INT8 resolution. Best suited for high-end setups where maximum possible precision is required.
`gemma-4-12b-heretic-Q6_K.gguf`	Q6_K	20GB–24GB VRAM (RTX 3090 / Dual GPUs)	The Near-Lossless Sweet Spot. Offers an incredibly high quality retention while saving significant space over the raw weights.
`gemma-4-12b-heretic-Q5_K_M.gguf`	Q5_K_M	16GB+ VRAM (RTX 4080 / 3090 / 4090)	The Enthusiast Tier. The highest fidelity compression level before hitting severe diminishing returns. Retains near-flawless parity with the source weights.
`gemma-4-12b-heretic-Q4_K_M.gguf`	Q4_K_M	16GB VRAM (RTX 4080 / 4070 Ti Super)	The Classic Standard. Features a balanced distribution of bit weights. Outstanding general purpose reasoning, chat response times, and stability.
`gemma-4-12b-heretic-IQ4_XS.gguf`	IQ4_XS	12GB–16GB VRAM (RTX 4070 / 4080)	The Optimizer's Choice. Custom tailored to maximize token throughput while protecting memory headroom. Fits alongside large context windows (up to 131k+) and MCP pipelines without aggressive system RAM spilling.
`gemma-4-12b-heretic-IQ3_XS.gguf`	IQ3_XS	8GB–12GB VRAM / Integrated Graphics / Mobile	The Lightweight Champion. Drastically reduces the memory footprint to run on lower-spec hardware, laptops, or mobile backends. The iMatrix lifeline keeps it coherent where flat 3-bit models collapse.

🧠 Technical Methodology: The iMatrix Advantage

When compressing a model down to 3, 4, or 5 bits, a basic, uncalibrated static quantization acts like a global color reduction on an image because it destroys critical structural details. Large Language Models contain sensitive outlier weights that dictate logic patterns, punctuation structure, and instruction adherence. Indiscriminately compressing these weights leads to specialization collapse, which causes a model to loop tokens, output formatting gibberish, or hallucinate heavily.

To protect this repo from degradation, these files were compiled using a Custom Hybrid Importance Matrix (imatrix.dat) strategy.

1. Calibration Profile

The calibration matrix was generated over a high-entropy, multi-domain text dataset specifically balanced for the local LLM stack:

Core Logic & Reasoning: Grounded using community standard English linguistic corpuses (text_en_large) to lock down general intelligence, deductive consistency, and vocabulary breadth.
Agentic Punctuation: Interwoven with structured tool-calling strings (tools_large) to safeguard brackets, colons, nested variables, and JSON structures necessary for automated file parsing pipelines like VaultForge.
Narrative Fidelity: Infused with creative prose samples to explicitly train the importance matrix to prioritize complex character interiority and vivid world-building pacing.

2. Execution Parameters

To avoid calculation errors or PCIe data-transfer distortions, the matrix was evaluated over a tight, isolated memory profile:

Total Cycles: 200 high-entropy chunks (representing roughly 100,000 tokens of diverse data)
Chunk Boundary: 512 tokens
Synchronization Sizing: -b 512 -ub 512 (Forcing physical batch alignment to protect mathematical scaling)

The resulting map acts as an explicit instruction booklet during quantization. It forces the llama-quantize tool to fiercely protect sensitive reasoning neurons while aggressively compressing the easy background language weights.

⚙️ Recommended Inference Settings

The Gemma-4 family is a highly capable but precise architecture. To avoid text stuttering, formatting drops, or interface crashes in backends like LM Studio, AnythingLLM, or llama-server, implement the following configurations:

1. Multi-Modal Vision Execution

Gemma-4 features a unified, encoder-free architecture, projecting raw image patches straight into the embedding layers. To handle vision tasks inside llama.cpp based frontends, you must load the companion multimodal projector file alongside your chosen text quant.

The verified companion file gemma-4-12b-it-mmproj-f16.gguf is hosted right here in the repository root. Ensure you map this file inside your backend Vision Adapter settings slot to seamlessly initialize image ingestion.

📜 Acknowledgements

Google DeepMind for pioneering the unified Gemma-4 architecture.
Philipp Emanuel Weidmann for developing the underlying Heretic abliteration framework.
Massive thanks to the open-source local AI community for continuously pushing the boundaries of what is possible on local consumer hardware.

⚠️ Disclaimer & Boundary Limits

This model is completely unaligned. It will output text without filtering, judgment, or warning labels. By downloading this model, you accept full responsibility for the prompts fed to it and the text generated by it. Use responsibly within local sandbox development setups.

🎛️ Streamlined Jinja Chat Template

If you encounter interface parsing issues with heavy multi-turn configurations or want to maximize token efficiency during rapid back-and-forth chat sessions, use this clean, hyper-efficient template:

jinja2

{%- for message in messages %}
<|turn|>{{ message['role'] }}
{{ message['content'] }}<|turn|>
{%- endfor %}
{%- if add_generation_prompt %}
<|turn|>assistant<|channel>thought <channel|>
{%- endif %}

culturerevolt/gemma-4-12b-heretic-abliterated-GGUF

Gemma-4-12B-Heretic-Abliterated-iMatrix-GGUF (Complete Suite)