You can deploy this model and start earning money today!

AutomatosX/
AX-Gemma-4-12B-IT-MLX-QAT-OptiQ-4bit-Assistant-MTP

Name: AX-Gemma-4-12B-IT-MLX-QAT-OptiQ-4bit-Assistant-MTP
Availability: InStock

Views

No views yet

AX Gemma 4 12B IT MLX QAT OptiQ 4-bit Assistant MTP

Parameter count: approximately 11.96B logical target parameters (12B class). 4-bit is the target quantization precision, not a 4B model-size claim. The separately packaged assistant is an acceleration component and is not included in the target count.

This is a self-contained MLX package for Apple Silicon. It combines the pinned upstream QAT OptiQ target with its paired 4-bit Gemma assistant for AX Engine multi-token prediction (MTP) / speculative decoding.

Try this model locally with AX Engine.

Attribution and changes

AutomatosX did not train Gemma 4, produce the QAT checkpoint, create the OptiQ quantization, or train the assistant. The target comes from mlx-community/gemma-4-12B-it-qat-OptiQ-4bit at revision 63912b888c04ba2c555f198685d10b05f54cf564. The assistant comes from mlx-community/gemma-4-12B-it-qat-assistant-4bit at revision 37ae18bbbc2c569d1c9ff6a5ca9359051f522df4.

AutomatosX packaged the pinned target and assistant together, normalized the assistant model_type for the AX runtime, generated native manifests, and added the exact-pairing contract. The target tokenizer is applied to the assistant subtree so both sides use identical token IDs. The target and assistant weights themselves are unchanged. The upstream OptiQ card is preserved as UPSTREAM_README.md.

No new training or benchmark results are claimed by AutomatosX.

Package details

Property	Value
Target format	MLX Safetensors
Official QAT base	`google/gemma-4-12B-it-qat-q4_0-unquantized`
Target quantization	OptiQ mixed 4/8-bit, group size 64
OptiQ allocation	171 components at 4-bit; 157 at 8-bit
Achieved target BPW	5.2453
Assistant precision	4-bit affine, group size 64
Pairing	Exact
Maximum draft depth	2
Configured context	262,144 tokens
Intended hardware	Apple Silicon

Download and serve

bash

hf download AutomatosX/AX-Gemma-4-12B-IT-MLX-QAT-OptiQ-4bit-Assistant-MTP \
  --local-dir ./AX-Gemma-4-12B-IT-MLX-QAT-OptiQ-4bit-Assistant-MTP

ax-engine doctor \
  --mlx-model-artifacts-dir ./AX-Gemma-4-12B-IT-MLX-QAT-OptiQ-4bit-Assistant-MTP

ax-engine serve ./AX-Gemma-4-12B-IT-MLX-QAT-OptiQ-4bit-Assistant-MTP --port 31418

The download is approximately 8.7 GB. AX Engine reads ax_gemma4_assistant_mtp.json, loads the drafter from assistant/, and uses the target model to verify every accepted token. Do not use assistant/ by itself as a general-purpose chat model.

The server exposes an OpenAI-compatible API. Consult the AX Engine repository for installation and API examples.

Validation and provenance

Validated on macOS arm64 with AX Engine 6.9.0 on 2026-07-20:

AX artifact doctor: ready, with no model issues
Safetensors headers, data bounds, and target/assistant index mappings: passed
Target weight shards: byte-exact against the pinned OptiQ source
Assistant weight: byte-exact against the pinned assistant source
Pairing contract: exact

See ax_provenance.json for immutable source revisions, transformations, and artifact hashes.

Quantization can change model quality, and speculative-decoding acceptance depends on the workload. Evaluate this package on your own tasks.

License

Apache License 2.0. See LICENSE, the Gemma 4 license page, and the official Google model cards for limitations and responsible-use guidance.