NVFP4 GGUF quantization of
llmfan46/gemma-4-12B-it-uncensored-heretic - an uncensored/heretic (abliterated) finetune of Google's Gemma 4 12B with vision support.
NVFP4 is NVIDIA's native 4-bit floating point format (E4M3) designed for Blackwell architecture GPUs (RTX 50-series, B100/B200). It provides:
This is an abliterated (uncensored/heretic) finetune of Google's Gemma 4 12B, a multimodal model with both text and vision capabilities. The original model was finetuned to remove safety alignment restrictions while maintaining the model's core capabilities.
Gemma 4 features a hybrid attention architecture with alternating sliding window and full attention layers, native vision encoding, and tool calling support.
1# Text only
2./llama-cli -m gemma4-12b-heretic-nvfp4.gguf -p "Hello" -n 100
3
4# With vision (requires mmproj)
5./llama-server -m gemma4-12b-heretic-nvfp4.gguf \
6 --mmproj mmproj-gemma-4-12b-heretic-f16.gguf \
7 --host 0.0.0.0 --port 8080 -ngl 99
1from huggingface_hub import hf_hub_download
2
3model_path = hf_hub_download(
4 repo_id="FreedomAISVR/Gemma-4-12B-it-Uncensored-Heretic-NVFP4-GGUF",
5 filename="gemma4-12b-heretic-nvfp4.gguf"
6)
7mmproj_path = hf_hub_download(
8 repo_id="FreedomAISVR/Gemma-4-12B-it-Uncensored-Heretic-NVFP4-GGUF",
9 filename="mmproj-gemma-4-12b-heretic-f16.gguf"
10)
