*Wikipedia-Korean perplexity, lower is better. Q4_K_M = 5.79 baseline. English builds are tuned on English; see each repo.
🍎 Why MLX for Korean but GGUF for English on iPhone? Apple-native MLX only does uniform quantization. Korean survives it (96 experts hold up); English needs our proprietary quantization, which only GGUF supports — so the English iPhone build ships as a GGUF you run with PocketPal. Honest, not lazy.
🆕 POCKET-26B — a Gemma4-26B-A4B-based sibling that loads in any app today (Ollama · LM Studio · PocketPal · MLX), no bleeding-edge runtime needed: GGUF (Q2_K 11 GB · Q4_K_M 17 GB · GPQA-Diamond 67%). Universal compatibility for 12 GB phones, PC, and browser.
Speed vs Bonsai
Benchmarks — what is measured, what is not
We measure Bonsai on the same machine with the same stock llama.cpp, and we tell you where we lose.
[measured] Generation speed — POCKET wins on both CPU and GPU:
POCKET-35B IQ1_M
Bonsai-27B Q1_0
CPU generate (Xeon, 16t)
27.0 tok/s
10.1
🟢 2.69×
GPU generate (H100)
197 tok/s
89
🟢 2.22×
GPU prompt (H100)
753
1816
🔴 0.41×
Quality (HellaSwag, 400q)
61.0%
60.0%
⚪ tie (CI overlaps)
[measured on a MacBook M3 Pro, 18 GB] — and on a laptop, POCKET wins every axis, including prompt processing:
POCKET-35B IQ1_M
Bonsai-27B Q1_0
Metal generate (tg64)
25.4 tok/s
12.8
🟢 1.99×
CPU generate (8 threads)
13.8 tok/s
4.4
🟢 3.13×
Metal prompt (pp128)
240.7 tok/s
73.4
🟢 3.28×
CPU prompt (pp128)
45.5 tok/s
9.6
🟢 4.75×
On a laptop GPU the arithmetic headroom that let Bonsai win prefill on an H100 is gone, so MoE sparsity wins across the board. POCKET-35B-Q2_K runs on the M3 Pro's CPU at 19.5 tok/s — on an 18 GB Mac, run Q2_K on CPU (-ngl 0); its 13 GB exceeds the recommended Metal budget.
[measured — GPQA Diamond, 198q, greedy] reasoning quality vs quantization:
Model
GPQA-Diamond (greedy)
Qwen3.6-35B-A3B
73.2%
POCKET-35B Q4_K_M
68.7%
POCKET-35B Q2_K
60.1%
[pending — community reports welcome] on-device iPhone and Strix Halo throughput. We publish only what we ran ourselves; help us fill the rest.
The same-size rival Ternary-Bonsai-27B-Q2_0 (7.2 GB) fails to load in upstream llama.cpp — it needs the PrismML fork. POCKET runs on the tools you already have.
Files in this repo
File
Size
bpw
Runs on
Korean PPL
POCKET-35B-Q4_K_M.gguf
21 GB
4.5
PC 32 GB RAM
5.79 (top)
POCKET-35B-Q3_K_M.gguf
16 GB
3.4
PC 24 GB
6.06
POCKET-35B-Q2_K.gguf ⭐
13 GB
2.6
mini-PC 16–24 GB
6.49 (best value)
POCKET-35B-IQ1_M.gguf
8.2 GB
1.9
16 GB RAM
9.69 (smallest)
Quickstart — no fork needed
bash
1# any recent llama.cpp — brew / winget / apt, or LM Studio / Ollama2llama-cli -m POCKET-35B-Q2_K.gguf -p "안녕하세요" -ngl 0 -t 83# reproduce our CPU numbers:4llama-bench -m POCKET-35B-IQ1_M.gguf -p 128 -n 64 -ngl 0 -t 16
Use physical-core count for -t (max ~32). Do not pass all threads — it can slow down sharply.
Lineage — where POCKET comes from
POCKET is quantized from Darwin-36B-Opus, VIDRAFT's flagship — a model bred and evolved over several generations on the Darwin platform (crossbreeding, healing, expert surgery). Darwin-36B-Opus itself traces back to a Qwen3.5-family MoE architecture.
Component
Origin
Starting checkpoint
Darwin-36B-Opus — VIDRAFT, multi-generation Darwin evolution
The CPU/GPU speed comes from the sparse-MoE architecture plus ordinary quantization — reproducible with the same base and the same tools. What we add is the Darwin-evolved weights, the honest measurement, the Korean tuning, and the pruning that makes the 5 GB phone builds.
Limitations
The iPhone/Mac speed is not yet measured by us — community reports welcome.
Extreme quants (IQ1_M) hurt Korean ~2.8× more than English; use Q2_K or larger for quality.
English phone builds trade quality for size; the PC build (PC-mix) is much closer to full quality.
License
Apache-2.0.
POCKET is a VIDRAFT model family. 35B, in your pocket. No GPU.
-0f6e56){kind=icon}
