This is not a standard full-precision Transformers checkpoint. The Hugging Face model page may show generic Transformers/vLLM snippets automatically, but this upload is intended for EXL3-compatible loaders.

Runtime validation

Local runtime smoke testing completed successfully on an RTX 4090 24GB.

Validated behavior:

Model loaded successfully in TabbyAPI.
VRAM after load was approximately 7.2 GB.
Short generation passed.
Reasoning/math prompt passed.
Instruction-following prompt passed.
Longer stability prompt passed.
Streaming response passed.
AI-Orchestrator integration passed.
No repeated text artifacts observed.
No broken Unicode/control-character artifacts observed.
No empty outputs observed.
No malformed streaming observed.

Reasoning / thinking support

The included chat_template.jinja is configured so thinking/reasoning mode can emit reasoning blocks separately from the final answer in compatible runtimes.

In local testing, reasoning was split successfully into reasoning metadata/panel output while final answers remained coherent.

Compatibility notes

During local testing, current ExLlamaV3/TabbyAPI code required fallback handling for the model's high GQA ratio in some layers.

Observed issue in unpatched fast path:

GQA ratio 16 exceeds G_MAX=8

Local workaround used:

Skip the incompatible bighead_scalar fast path when GQA ratio is greater than 8.
Allow fallback attention paths such as SDPA/Triton/PyTorch attention.

If your loader fails with a GQA ratio error, update your ExLlamaV3/TabbyAPI build or apply equivalent fallback behavior.

Source model

Base/source model:

igorls/gemma-4-12B-it-heretic

Please follow the source model license and terms.

sjoe1244/gemma-4-12B-it-heretic-exl3-4bpw

gemma-4-12B-it-heretic EXL3 4-bit

Quantization

Recommended loader

Runtime validation

Reasoning / thinking support

Compatibility notes

Source model

sjoe1244/
gemma-4-12B-it-heretic-exl3-4bpw