If your machine has no discrete GPU (or you want to keep the GPU free), these are the
models and setups that actually work — with real token/s numbers, not promises.
⭐ The star: Maple Preview 20B-A1B (ternary 2-bit)
The fastest useful model on a CPU we have found — over 28 tokens/s on a 4-core laptop.
Model
Size
Quant
CPU decode (thinkpad2, 4 threads)
Maple Preview (20B-A1B, 256-expert MoE, 8 active)
5.5 GiB
TQ2_0 ternary, 2.06 bpw
33.9 t/s (tg128) · 28.2 t/s (benchy tg64)
Maple Preview is DeepGrove's open-source reasoning model, designed from the start for
efficient on-device inference (24 layers, 3:1 SWA-512:GA attention, 131k context, MIT
license). It is the real star of this collection: on our CPU it runs at
28–34 tokens/s — comfortably interactive — while its 20B total / 1B-active ternary
weights keep it to a 5.5 GB file that fits any machine with 16 GB of RAM.
Reference points (from the DeepGrove team and our own measurements):
this CPU (i5-8350U, 4 threads): prompt 512 tok → 100.8 t/s · decode tg128 → 33.9 t/s
Apple M2 Max CPU: ~360 t/s prompt · ~77 t/s decode
Apple M4 mini (DeepGrove's measurement): 200+ t/s
The engine is the DeepGrove llama.cpp fork (the maple architecture + TQ2_0
support); the GGUF we use is their maple-preview-TQ2_0-head-Q4_K.gguf.
1llama-server -m maple-preview-TQ2_0-head-Q4_K.gguf \2 --ctx-size 131072 --cache-type-k q8_0 --cache-type-v q8_0 \3 --threads 4# 4 beats 8 on this CPU (33.9 vs 21.8 t/s)
LiquidAI LFM2.5 family — mixed 4-bit GGUFs by ljupco
The LFM2.5 models (2.6B dense, 1.2B-Thinking, 8B-A1B MoE) with mixed quantizations:
the bulk of the weights at 4-bit, the most sensitive tensors kept at higher precision.
These numbers come from a systematic porting and benchmarking exploration across three
engines — stock llama.cpp (with the DeepGrove fork for Maple), ik-llama.cpp, and
vllm.cpp — including kernel-level work (fused ops, integer-dot kernels, a ternary
gemv) and a detailed analysis of why the DeepGrove 8x8 gemv cannot run on
standard-quantized weights:
This collection is entirely built on the work of others, and we are deeply grateful:
DeepGrove AI — for the Maple Preview model, the TQ2_0 ternary quantization, and
their llama.cpp fork with the Maple architecture support. The ternary design and the
on-device focus are what make 28+ t/s on a CPU possible. Thank you!
Liquid AI — for the LFM2.5 family and its gated-delta / shortconv architecture,
and for publishing the weights openly. Thank you!
llama.cpp / ggml — the core inference engine and its maintainers and contributors.
The HuggingFace / GGUF ecosystem for the format, the tooling, and the platform.
Any remaining errors are ours. Benchmark numbers are single-machine measurements;
expect ±10–20% day-to-day noise.
