model loading in mlx-vlm: passed
local conversion / quantization: passed
text generation smoke test: mixed
image generation path smoke test: mixed
lm studio 0.4.11+1 loading: not supported yet
notes: Conversion finished successfully, but Gemma 4 runtime behavior still depends on current mlx-vlm and LM Studio support. In local checks, the models load in mlx-vlm and can emit tokens, but output quality still needs tuning, and LM Studio 0.4.11+1 currently refuses to load Gemma 4.

Files

Important files in this repo:

config.json
generation_config.json
chat_template.jinja
processor_config.json
tokenizer.json
tokenizer_config.json
model.safetensors.index.json
model-*.safetensors

Usage

Text generation

bash

mlx_vlm.generate \
  --model /path/to/Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16 \
  --prompt "Describe local inference in one short sentence." \
  --max-tokens 128 \
  --temperature 1.0 \
  --trust-remote-code

Image prompt

bash

mlx_vlm.generate \
  --model /path/to/Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16 \
  --image /path/to/example.png \
  --prompt "Describe this image." \
  --max-tokens 128 \
  --temperature 1.0 \
  --trust-remote-code

LM Studio

LM Studio 0.4.11+1 detects these repos but currently refuses to load Gemma 4 with ValueError: Gemma 4 support is not ready yet, stay tuned!. Use mlx-vlm for now until LM Studio adds native Gemma 4 support.

Notes

This repo reflects a local conversion workflow and validation pass on Apple Silicon. Behavior can vary with mlx-vlm version, sampling parameters, and prompt style.

vanch007/Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16

Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16

Overview

Conversion Notes

Validation