michaellin/

gemma-4-12B-it-qat-4bit-gs32

This model is quantized from google/gemma-4-12B-it-qat-q4_0-unquantized using the command python -m mlx_vlm convert --hf-path google/gemma-4-12B-it-qat-q4_0-unquantized -q --q-group-size 32 --q-bits 4 --mlx-path "./gemma-4-12B-it-qat-4bit-gs32". It uses the correct group size of 32, which corresponds to the 32 nibbles (uint4) of Q4_0.

Simply put, the default of mlx-lm and mlx-vlm is 64 group size, which does not match the QAT parameters that Google is using, since that targets llama.cpp, which uses 32 by default. This quantization fixes that at the cost of slightly inflated quant size (scale parameter storage size is effectively doubled).