Variant	Quantization	Size	HF Repo	Use Case
BF16	None	22 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-BF16`	Text generation
Q8	8-bit	12 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-Q8`	Text generation
Q6	6-bit	8.9 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-Q6`	Text generation
Q4	4-bit	6.2 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-Q4`	Text generation

Vision-Capable Variants (mlx-vlm)

Variant	Quantization	Size	HF Repo	Use Case
BF16	None	25 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-VLM-BF16`	Image + Text
Q8	8-bit	13 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-VLM-Q8`	Image + Text
Q6	6-bit	11 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-VLM-Q6`	Image + Text
Q4	4-bit	7.5 GB	`tg-rising/gemma-3-12b-it-heretic-v2-MLX-VLM-Q4`	Image + Text

Requirements

pip install -U mlx-vlm

Usage - Image Understanding

CLI

bash

# Analyze an image
python -m mlx_vlm.generate \
  --model tg-rising/gemma-3-12b-it-heretic-v2-MLX-VLM-Q4 \
  --prompt "Describe this image in detail." \
  --image /path/to/image.jpg \
  --max-tokens 100

Python API

python

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load model
model_path = "tg-rising/gemma-3-12b-it-heretic-v2-MLX-VLM-Q4"
model, processor = load(model_path)
config = load_config(model_path)

# Prepare image + text
image = ["/path/to/image.jpg"]
prompt = "What is in this image?"

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=len(image)
)

# Generate
output = generate(model, processor, formatted_prompt, image, max_tokens=100)
print(output)

Quantization Details

Bits: 4-bit
Mode: affine
Group Size: 64
Vision Components: Preserved (not quantized)
Text Components: 4-bit

License

This model is subject to the Gemma license.