gemma-4-31B-uncensored-heretic · MLX 8-bit
MLX conversion of llmfan46/gemma-4-31B-it-uncensored-heretic, a fine-tune of Google's Gemma 4 31B Instruct. Quantized to ~8.6 bits per weight using mlx-vlm v0.4.3 on Apple Silicon.
Performance on Apple M4 Max · 128 GB
Peak memory: ~34 GB
Prompt throughput: ~20.6 tok/s
Generation speed: ~14.5 tok/s
Requirements
bashpip install -U mlx-vlm
Gemma 4 support requires mlx-vlm >= 0.4.3. Standard mlx-lm does not yet support the gemma4 architecture.
Usage
Text only
bashpython -m mlx_vlm generate
--model TxemAI/gemma-4-31B-uncensored-heretic-mlx-8bit
--prompt "Your prompt here"
--max-tokens 512
With image
bashpython -m mlx_vlm generate
--model TxemAI/gemma-4-31B-uncensored-heretic-mlx-8bit
--prompt "Describe this image."
--image path/to/image.jpg
--max-tokens 512
Python API
pythonfrom mlx_vlm import load, generate
model, processor = load("TxemAI/gemma-4-31B-uncensored-heretic-mlx-8bit")
response = generate(
model,
processor,
prompt="Your prompt here",
max_tokens=512,
temperature=0.7,
)
print(response)
Memory requirements
PrecisionVRAMBF16 (full)~62 GBQ8 (this model)34 GBQ418 GB
Notes
The model activates Gemma 4's thinking channel (<|channel>thought) on reasoning-heavy prompts — this is expected behaviour.
The mel filter warning on load is harmless; it relates to the audio encoder and does not affect text or vision inference.
Unofficial community conversion. For the original fine-tune see llmfan46/gemma-4-31B-it-uncensored-heretic.
Conversion
bashpython -m mlx_vlm convert
--hf-path llmfan46/gemma-4-31B-it-uncensored-heretic
--mlx-path ./gemma-4-31B-uncensored-heretic-mlx-8bit
--quantize --q-bits 8
Credits
Google DeepMind — Gemma 4 base model
llmfan46 — uncensored-heretic fine-tune
ml-explore — MLX framework
Blaizzy — mlx-vlm library
