You can deploy this model and start earning money today!

pipenetwork/
Gemma-4-26B-A4B-it-MLX-5bit

Name: Gemma-4-26B-A4B-it-MLX-5bit
Availability: InStock

google/gemma-4-26B-A4B-it

quantized

apache-2.0

5-bit

Views

No views yet

Gemma-4-26B-A4B-it-MLX-5bit

MLX (Apple Silicon) conversion of google/gemma-4-26B-A4B-it (Mixture-of-Experts, ~4B active), quantized to 5-bit. Text-only build.

Quantizations

Part of the Gemma-4-26B-A4B-it MLX collection.

Variant	Notes
4-bit	4-bit · community build (mlx-community)
8-bit	8-bit · near-lossless
6-bit	6-bit · high quality
5-bit (this repo)	5-bit

Use with mlx-lm

bash

pip install mlx-lm
python -m mlx_lm generate --model pipenetwork/Gemma-4-26B-A4B-it-MLX-5bit --prompt "Explain Mixture-of-Experts briefly." -m 256

Validation

Smoke-tested locally: loads and generates coherent text.

License

Apache 2.0 (inherited from base). Quantization config:

{"group_size": 64, "bits": 5, "mode": "affine", "language_model.model.layers.0.router.proj": {"group_size": 64, "bits": 8}, "language_model.model.layers.1.router.proj": {"group_size": 64, "bits": 8},