Gemma4 MTPLX Optimized Speed
This is an MTPLX pair bundle for Gemma 4 31B speculative decoding on Apple Silicon.
It is not a single vanilla Transformers model directory. The repository contains two MLX-format artifacts:
target/ - Gemma 4 31B IT target, MLX Q4 affine group-size 64assistant/ - official Gemma 4 31B assistant drafter, MLX Q6 affine group-size 64
Use this pair when absolute throughput is the priority.
Source
- Target source:
google/gemma-4-31B-it
- Target revision:
145dc2508c480a64b47242f160d286cff94a2343
- Assistant source:
google/gemma-4-31B-it-assistant
- Assistant revision:
cffbbd2cea41ea56a0fa5b0487e0d445121fd204
Both artifacts were converted locally to MLX format.
Quantization
Target:
1bits: 4
2group_size: 64
3mode: affine
Assistant:
1bits: 6
2group_size: 64
3mode: affine
MTPLX Usage
After downloading this repository, point MTPLX at the two subdirectories:
1mtplx bench gemma-mtp \
2 --target-model ./target \
3 --assistant-model ./assistant \
4 --prompt-suite mtplx/benchmarks/prompts/flappy.jsonl \
5 --max-tokens 1000 \
6 --draft-block-sizes 6 \
7 --allow-unverified-gemma
The Gemma 4 assistant is a separate drafter model. MTPLX uses exact speculative sampling with target verification and residual correction.
Local Benchmark
Prompt: single-file HTML5 Canvas Flappy Bird game, capped at 1000 generated tokens.
Sampler:
1temperature: 1.0
2top_p: 0.95
3top_k: 64
4seed: 0
Best observed block size:
1block_size: 6
2acceptance: 830 / 846 = 98.11%
Observed MTPLX throughput samples:
143.56 tok/s
244.46 tok/s
344.07 tok/s
The bundled benchmark JSON files are in benchmarks/.
Notes
This release is optimized for MTPLX speed experiments. For a higher-precision target, use Youssofal/Gemma4-MTPLX-Optimized-Quality.
Gemma 4 is released by Google under the Gemma 4 license terms linked above.
