All three packs generate text on GPU: maple-f16.gguf (~385 tok/s, H200), maple-q4_k_m.gguf (GPU-verified), and maple-tq2_0.gguf (~97 tok/s on an RTX 4000 Ada; previously CPU-only)**, with the Maple-enabled llama.cpp fork at github.com/stamsam/llama.cpp (branch prism, rev 9ee03ee) — all layers GPU-offloaded.

Mainline llama.cpp CANNOT run any of these files. The maple architecture exists only in the stamsam/llama.cpp fork (the PrismML upstream does not have it), and the fork's ternary format is not interchangeable with mainline types.

Development artifacts of an ongoing port, shared openly.

Overview

This repository hosts GGUF conversions of deepgrove/maple-preview — a 20B-A1B ternary-weight Mixture-of-Experts reasoning LLM by DeepGrove (2026, MIT license) — plus the original BF16 checkpoint (in bf16-checkpoint/). The GGUF pipeline was built on the PrismML llama.cpp fork; the complete Maple runtime (converter + inference graph) lives in the stamsam/llama.cpp fork (branch prism).

Model summary

Property	Value
Base model	deepgrove/maple-preview (MIT)
Family	Maple-Preview — 20B total params, ~1B active (A1B)
Layers	24
Hidden size	2048
Attention	GQA — 16 query heads, 4 KV heads, head_dim 128
Attention pattern	3:1 hybrid — SWA-512 (sliding window 512) : Global Attention (GA)
RoPE	Partial (64/128 dims, theta 10000) on SWA layers; no RoPE at all on GA layers
QK structure	Flash-head QK — per-head RMSNorm (q_norm/k_norm), k_proj at 512 dims
MoE	256 experts, 8 active; moe_intermediate 512; clamp-7 SwiGLU; fp32 router renorm
Context length	131072
Vocab size	151936 (no weight tying; no bias terms)
RMSNorm eps	1e-6
Stored weights	Per-row-scaled ternary: W ≈ alpha_row * T, T in {-1,0,1} (~39% exact zeros) for experts and q/o projections; k_proj, lm_head, embeddings, router, norms stored denser
Checkpoint	~40.4 GB BF16 · 20.214 B params · 18,651 tensors

Hosted files

File	Size	Notes
`maple-q4_k_m.gguf`	12.33 GB (measured)	Uniform Q4_K_M — runs (GPU)
`maple-tq2_0.gguf`	5.45 GB (measured)	Tiered: 168x fork ternary `tq2_0` (type 35) + 2x Q4_0 + 121x F32 — runs (GPU, ~97 tok/s RTX 4000)
`maple-f16.gguf`	40.5 GB (measured)	Dense F16 reference; 291 GGUF tensors (18,651 HF tensors merged)
`bf16-checkpoint/`	40.4 GB (20 files)	Original BF16 checkpoint — 9 safetensors shards + configs + tokenizer + modeling code

Tensor counts and shapes were verified by post-write inspection (GGUFReader): 291 tensors, architecture key maple, per-tensor types confirmed (f16: F16/F32; tq2_0: 168x type-35 + 2x Q4_0 + 121x F32; q4_k_m: 145x Q4_K + 25x Q6_K + 121x F32).

Status

The Maple compute graph is implemented and verified in github.com/stamsam/llama.cpp (branch prism). All three packs generate coherent text (tested: "What is 2+2?" → "The answer is 4."): maple-f16.gguf and maple-q4_k_m.gguf with all layers offloaded to an NVIDIA H200, and maple-tq2_0.gguf on an RTX 4000 Ada at ~97 tok/s (CPU: ~5.5 tok/s) and on a Jetson Xavier at ~14.8 tok/s.

What the graph implements (matching the MLX reference deepgrove/maple-preview-2bit-mlx):

Hybrid attention — 3:1 SWA-512:GA: sliding-window layers (window 512, partial 64/128-dim RoPE, theta 10000) with full-attention layers every 4th (il % 4 == 3) that carry no RoPE (n_rot = 0).
Flash-head QK — per-head RMSNorm (q_norm/k_norm) on the reshaped Q/K before RoPE; k_proj at 512 dims (4 KV heads x 128).
MoE — 256 experts, top-8, moe_intermediate 512, clamp-7 SwiGLU (silu(min(gate, +7)) * clip(up, -7, +7)), fp32 softmax + renormalized routing.
KV — sliding-window KV cache (window 512) for the SWA layers.

Known limitations:

Ternary GPU support is new — tq2_0 (GGML type 35) got its CUDA kernels (dequant, get_rows, mmvq vec_dot) at fork rev 9ee03ee; big-batch matmuls use the dequant+gemm path (mmq kernels not yet ported).
Fork-only — the maple arch lives in stamsam/llama.cpp (branch prism); mainline llama.cpp and the PrismML upstream fork cannot load these files.
Early validation — verified for coherent generation on simple prompts; no benchmarks, perplexity, or systematic evals yet.

For reference implementations of the model itself:

Apple Silicon: deepgrove/maple-preview-2bit-mlx (official MLX runtime)
Elsewhere: the original PyTorch checkpoint (deepgrove/maple-preview) — or the bf16-checkpoint/ copy in this repo

Usage

These files only work with stamsam/llama.cpp (branch prism); mainline llama.cpp cannot load them.

bash

# GPU (f16 or Q4_K_M packs)
llama-cli -m maple-q4_k_m.gguf -p "Your prompt here" -n 512

# Server (OpenAI-compatible API)
llama-server -m maple-q4_k_m.gguf --port 8080

maple-tq2_0.gguf runs the same way but executes on the CPU backend until CUDA kernels for the ternary type land. No Maple-specific CLI flags are required.

Quantization recipe

All steps run CPU-side on an NVIDIA H200 host: the converter and quantizer are CPU processes — no GPU compute is involved, and while CUDA is available to torch, it is unused in these steps.

Convert: convert_hf_to_gguf.py (PrismML fork, with the Maple converter in this fork) on the deepgrove/maple-preview checkpoint → maple-f16.gguf (dense reference).
Quantize: llama-quantize on maple-f16.gguf:
- maple-tq2_0.gguf — per-tensor type mix mirroring the official maple-preview-2bit-mlx tiering:
  - fork ternary tq2_0 (type 35, ~2.06 bpw) for attn_q/attn_k/attn_v/attn_output and all ffn_*_exps;
  - Q4_0 for token_embd and output (lm_head);
  - F32/F16 for routers and norms.
- maple-q4_k_m.gguf — uniform Q4_K_M over the entire model.

The maple architecture is registered in the fork's arch table with a dedicated llama_model_maple class (hparams, tensor map, and full compute graph). Note: an earlier attempt used the fork's type-42 Q2_0 layout, which the quantizer can write but no inference kernel can read — it was superseded by the type-35 tq2_0 format used in maple-tq2_0.gguf.

Caveats

Fork-only formats, no interop: the fork's ternary tq2_0 (GGML type 35) and its Q2_0 variant (type 42) are custom layouts — mainline llama.cpp has different Q2_0/Q2_K types under the same names and cannot read these files. There is no compatibility layer.
CPU-only ternary: maple-tq2_0.gguf has no CUDA kernels yet — expect ~5.5 tok/s CPU throughput on a 96-core host, not H200 speed.
Q4 on ternary weights is wasteful: Maple's experts and q/o projections are already ternary in storage; a uniform Q4_K_M re-quantizes them to 4 bits with little precision benefit. maple-q4_k_m.gguf is a quantizer sanity check / compatibility data point, not the recommended format — the tiered maple-tq2_0.gguf mirrors the intended storage layout.
Not the MLX format: the tiering philosophy matches the official MLX pack (2-bit ternary for projections+experts at 16 codes/u32, 4-bit for lm_head+embeddings at 8 codes/u32, BF16 for router+norms), but the binary formats differ. Files are not interchangeable between the two runtimes.
Development artifacts, not a release: trial/experimental work on a custom fork; formats and tensor layouts may change without notice as the port progresses.

Development log

Converted deepgrove/maple-preview (BF16 checkpoint, 20.214 B params / 18,651 tensors) → maple-f16.gguf via the fork's converter (CPU, H200 host).
Quantized → maple-tq2_0.gguf (tiered ternary pack: 168x type-35, 2x Q4_0, 121x F32) and maple-q4_k_m.gguf (uniform Q4_K_M).
Post-write verification: tensor counts and shapes inspected against the source checkpoint (2026-08-04).
Graph implemented + verified (2026-08-04): hybrid SWA/GA attention, flash-head QK, 256-expert clamp-7 SwiGLU MoE, fp32 renorm routing — f16 and Q4_K_M packs generate coherent text on an H200 (~385 tok/s at f16, all layers GPU-offloaded).
Ternary GPU kernels (rev 9ee03ee): CUDA dequant + get_rows + mmvq vec_dot for tq2_0 — verified on an RTX 4000 Ada at ~97 tok/s decode / ~111 tok/s prefill (CPU: 5.5 tok/s).
Jetson Xavier verified (2026-08-04): maple-tq2_0.gguf runs CPU-only on the Xavier (aarch64, 8 cores, 14 GB RAM) at 14.8 tok/s decode / 37.5 tok/s prefill, coherent output.
Next phase: quality benchmarks vs the MLX reference, CUDA build for the Xavier (sm_72) once its toolkit is set up.

Credits

Model: DeepGrove — deepgrove/maple-preview (MIT)
Official on-device runtime: deepgrove/maple-preview-2bit-mlx
Base tooling: PrismML llama.cpp fork
Maple runtime fork + conversion + packaging: stamsam/llama.cpp — MIT

stamsam/maple-preview-gguf

⚠️ STATUS — READ THIS BEFORE DOWNLOADING