Hardware: M5 MacBook Air 32GB
Inference Framework: oMLX v0.5.1
Settings:
- Thinking: Disabled
- Chat template parameter: enable_thinking=false (forced)
- TurboQuant KV Cache: Disabled (if enable: reduces speed)
- Native MTP: Enabled (key speed improvement)

Performance Benchmarks

Note: Results are for reference only and may vary depending on hardware, software configuration, and workload.

Test	TTFT(ms)	TPOT(ms)	pp TPS	tg TPS	E2E(s)	Throughput	Peak Mem
pp1024/tg128	1469.4	22.13	696.9 tok/s	45.5 tok/s	4.304	267.6 tok/s	6.10 GB
pp4096/tg128	5871.5	22.67	697.6 tok/s	44.5 tok/s	8.778	481.2 tok/s	6.72 GB

Continuous Batching (pp1024 / tg128)

Batch	tg TPS	Speedup	pp TPS	pp TPS/req	TTFT(ms)	E2E(s)
1x	45.5 tok/s	1.00x	696.9 tok/s	696.9 tok/s	1469.4	4.304
2x	48.2 tok/s	1.06x	643.7 tok/s	321.9 tok/s	3181.7	8.491
4x	80.3 tok/s	1.76x	636.0 tok/s	159.0 tok/s	6278.2	12.816

Intelligence Benchmark

Note: Each benchmark round tests only 30 questions. Results are for reference only.

Benchmark	Accuracy	Correct	Total	Time(s)	Think
MMLU	70.0%	21	30	33.3	No
TRUTHFULQA	86.7%	26	30	12.1	No
GSM8K	96.7%	29	30	159.2	No
MATHQA	33.3%	10	30	17.3	No
HUMANEVAL	80.0%	24	30	136.3	No