6-bit MLX quantization of google/gemma-4-26B-A4B-it, for Apple Silicon (~20 GB). Sparse-MoE vision-language model — run it with mlx-vlm, not mlx-lm. The expert router (router.proj) is kept at 8-bit precision.

Usage

pip install -U mlx-vlm

bash

python -m mlx_vlm.generate \
  --model TyKaoz/gemma-4-26B-A4B-it-6bit \
  --prompt "Explique la quantization en une phrase." \
  --max-tokens 200

Base	Tool	Precision	Size
`google/gemma-4-26B-A4B-it`	`mlx-vlm`	6-bit · group 64 (router 8-bit)	~20 GB

By TyKaoz — privacy-first native macOS LLM chat client. Apache 2.0, inherited from the base model.