This repository contains an MLX-ready experimental QAT Q5 distribution of Google's Gemma 4 12B Unified instruction-tuned model, prepared for local inference on Apple Silicon Macs.
This is not a newly fine-tuned model and it is not an official Google QAT Q5 release. It is a quantized MLX redistribution converted from Google's official Quantization-Aware Training (QAT) unquantized checkpoint. No structural modifications or additional training have been applied to the model architecture.
Important Notice
This repository is hosted primarily as a dedicated engine source for the R.E.V.I.S. application ecosystem. You are free to download and use this model package for your own local MLX projects or workflows, subject to the Apache License 2.0 and Google's Gemma terms.
For the original model cards, architecture details, intended usage, limitations, and evaluation information, refer to the official upstream models:
This package uses MLX affine quantization with a mixed layout chosen to follow the tensor boundaries of Google's official QAT Q4 GGUF release as closely as practical in MLX, while raising the default quantized language tensors to Q5 for this experiment:
Default language tensor quantization: MLX affine Q5
Group size: 64
Mode: affine
Token embeddings: MLX affine Q6
Audio, vision, multimodal projection components, MoE routers where present, and other keep-list tensors: BF16
Norms, biases, and small non-linear tensors: preserved in their source floating-point representation
In practical terms, this is not a plain "quantize every linear layer to Q5" package. The embedding table is intentionally kept at higher precision, and the multimodal / routing side of the model is not aggressively quantized. This keeps the MLX package close to the official Google QAT Q4 tensor policy while testing whether Q5 gives a useful quality-size tradeoff.
Optimized for R.E.V.I.S. (Local Cognitive OS)
We host and test this model package to serve as a local reasoning and judgment engine for R.E.V.I.S.
R.E.V.I.S. is a 100% local Cognitive OS for Multi-Agentic AI. It transforms your Mac devices into a distributed Agentic Swarm via zero-config Wi-Fi clustering, allowing you to run heavy AI workloads like recursive web research, dynamic RAG generation, and multi-step logic without killing single-machine performance.
If you are interested in pushing the limits of local AI and open-weight models, check out our project.
Gemma 4 12B Unified is a practical middle point between the smaller E2B / E4B models and the larger 26B MoE / 31B dense models. This QAT Q5 build is experimental and should be compared against the corresponding QAT Q4 and regular Q5/Q6 builds before production use.
This repository redistributes a quantized derivative of Google's Gemma 4 instruction-tuned QAT release, which is distributed by Google under the Apache License 2.0.
This derivative is likewise distributed under the Apache License 2.0. A copy of the license is included in the LICENSE file in this repository, and can also be found at https://www.apache.org/licenses/LICENSE-2.0.
Modification Notice
Compared with the official Google QAT source checkpoint, this repository applies the following packaging modification:
The source checkpoint was converted to MLX format and quantized with a GGUF-like mixed Q5/Q6/BF16 policy for local MLX inference.
No fine-tuning, additional training, or architecture-level modification has been applied.
