An experimental MLX affine mixed-precision quantization of google/gemma-4-E2B-it. This is my first public model release and my first public quantization, so practical feedback from people who try it is especially welcome. It is intended for local text generation on Apple Silicon using MLX-LM; feedback and discussion are welcome through the maintainer account maglun.
This is a compressed model, not a claim of lossless behavior or a new quality frontier. In the same-harness release evaluation it retained broadly comparable instruction-following results, scored strongly on HumanEval+ and GSM8K, and showed a measurable MMLU-Pro reduction versus the bf16 H0 comparator.
The release uses the updated Google Gemma chat template from immutable release-metadata revision 3e22461f65e89153144f8adb70e3b8c2cc9845a7 (template SHA-256 0a2c8073c878ab1da004bee933a998606537bbb62016310352c7285c3f01c5b5), together with the thinking-enabled path. This is the newer release template rather than the older template embedded in the original quantized artifact. For programmatic use, use the standard mlx_lm APIs and the tokenizer files included here. The tested runtime was MLX 0.32.0, MLX-LM 0.31.3, Python 3.12.13, macOS 26.5, on an Apple M4 Pro with 64 GB memory.
Measured resources
These measurements are on the named Apple M4 Pro machine above, with thinking enabled, greedy temperature 0, batch size 1, and no intentional concurrent benchmark load. They are not universal speed claims.
Measurement
Result
Cold load, median (range; 3 fresh processes)
0.958 s (0.946–0.968 s)
2,048-token prompt; 256-token maximum; 182 generated before normal stop
2.278 s (2.274–2.280 s) end-to-end
Prefill rate, median (range)
4,007.98 tok/s (4,000.50–4,023.08 tok/s)
Decode rate, median (range)
109.99 tok/s (109.96–110.09 tok/s)
MLX peak memory, 2K run
2.779 GiB
Process peak RSS, 2K run
3.0690 GiB median (3.0688–3.0691 GiB)
8,192-token functional context peak
2.904 GiB MLX peak; 3.069 GiB RSS
The complete corrected trial record and machine context are retained in the private release evidence as p6-resource-measurement-v2.json. The earlier v1 record is superseded because its derived throughput included prompt prefill.
Evaluation
Release-time evaluation used frozen subsets, greedy temperature 0, thinking enabled, the same prompt/scorer harness for candidate and bf16 H0, and no uniform4bit comparator. The corrected machine-readable summary is included in eval/final-metrics-v2.json; raw per-item records are retained in private release evidence.
Task / metric
Candidate
bf16 H0
Difference
IFEval strict prompt
451/541 (83.36%)
455/541 (84.10%)
−0.74 pp
IFEval strict instruction
737/834 (88.37%)
744/834 (89.21%)
−0.84 pp
IFEval loose prompt
471/541 (87.06%)
466/541 (86.14%)
+0.92 pp
IFEval loose instruction
759/834 (91.01%)
757/834 (90.77%)
+0.24 pp
MMLU-Pro-400 accuracy
216/400 (54.00%)
236/400 (59.00%)
−5.00 pp
HumanEval+ pass@1
139/164 (84.76%)
135/164 (82.32%)
+2.44 pp
GSM8K-400 accuracy
359/400 (89.75%)
366/400 (91.50%)
−1.75 pp
Aggregate / average†
3132/3714 (84.33%)
3159/3714 (85.06%)
−0.68 pp average
IFEval and MMLU-Pro are frozen release subsets; their names and denominators should not be read as full-suite claims. Output/parser failures count as incorrect in the full frozen denominators. Operational outcomes included IFEval cap hits 3/1 candidate/H0, MMLU-Pro cap hits 4/2, HumanEval+ 0/0, and GSM8K 1/0; these are disclosed in eval/final-metrics-v2.json.
The quantization recipe, source provenance, file hashes, and package manifest are included with this release. The model is explicitly text-only: no image input, audio input, or multimodal processor is included.
How this quantization was made
The release follows this public lineage:
Start from the text-capable google/gemma-4-E2B-it checkpoint.
Fold the AWQ transformation into the extracted text model.
Allocate mixed precision using a layer policy that keeps more sensitive weights at higher precision and uses lower precision where appropriate.
Apply GPTQ-style rounding with a fixed calibration manifest and seed.
Convert the result into MLX affine safetensors for use with mlx-lm.
“GPTQ” here describes a rounding/build step in the lineage; this repository is an MLX model and is not an AutoGPTQ package. The project-local implementation attribution, exact source revisions, policy hash, calibration-manifest hash, seed, and output identity are recorded in quantization_recipe.json and release-manifest.json.
Limitations and license
Quantization can change model behavior, including long-form completion and benchmark outcomes. Hardware testing was performed primarily on one Apple M4 Pro configuration. This is an experimental research release with no product support and no safety fine-tuning beyond the upstream model. Do not infer broad safety, multilingual, or production-readiness claims from the included text evaluation.
The model is released under the Apache-2.0 terms applicable to the upstream model; see LICENSE. Gemma usage remains subject to Google's upstream terms and attribution requirements.
Citation and feedback
Please cite the upstream Gemma 4 E2B-it work and identify this repository and exact revision when reporting results. Feedback and corrections are welcome through the maglun maintainer account and repository discussions.
†Totals sum the displayed rows, while the difference is the unweighted average of the seven displayed percentage-point differences. The four IFEval views are counted separately; this descriptive aggregate is not a QRS or a single-suite score.
