Shohin (小品) is the bonsai class for small trees that keep the full character and
proportions of a full-size specimen — which is the intent here: 27B parameters, pruned and
trimmed, still writing working code.
3,151,296,128 bytes = 3.151 GB (2.935 GiB) — a 27B parameter model at 1.125 bits per
weight. 652 MB smaller than the 3.803 GB parent, or 1.58 GB smaller than the 4.73 GB
that LM Studio reports for the parent (which bundles the vision projector).
Stock Q1_0, so it loads in LM Studio and any recent llama.cpp with no patching.
Measured capability
pass@1 by execution on newfacade/LeetCodeDataset — each generated solution is run
against the problem's own asserts. Greedy, enable_thinking=False, n=150:
model
size
pass@1
95% CI
valid Python
this model
3.151 GB
14/150 = 9.3%
[4.7%, 14.0%]
88.0%
52-block variant
2.937 GB
3/150 = 2.0%
[0.0%, 4.2%]
77.3%
48-block variant
2.945 GB
0/40 = 0%
—
70–80%
56 blocks is the shallowest depth that still writes working code. The gap to 52 blocks
is real and significant (z=2.75, p=0.006); below 56 the model collapses.
The parent measured 17.5% at n=40 (95% CI [5.7%, 29.3%]) but has not been measured at
n=150, so the gap to the parent is not quantified here. Treat 9.3% as this model's own
number, not as a claim of parity.
How it was built
1. Depth prune 64 → 56 blocks, dropping groups 5 and 7 (original blocks 20–23, 28–31).
Chosen by leave-one-out perplexity then greedy search — placement matters enormously, with
a 6.9× perplexity spread between best and worst 16-block choices at identical file size.
Layer type derives from index mod 4 (qwen35.full_attention_interval = 4), so only whole
4-block groups can be removed without breaking Bonsai's 3:1 linear/full attention phase.
Leave-one-out also found 10 of the 16 groups individually improve perplexity when
removed at this quantisation — the model is over-deep for 1.125 bpw. Two are absolute
exceptions: dropping group 0 gives perplexity 14,592; group 15 gives 365.
2. Vocabulary 248,320 → 98,782. Across 11.9 M tokens of real traffic (source code,
wikitext-2, SWE-smith agent trajectories, LeetCode statements) only 66,741 distinct
tokens — 26.9% — were ever used. Kept those plus the lowest-numbered unused ids as
headroom, then took a transitive closure over the merge table (+478 ids) so every
surviving token is still reachable by BPE. Without that closure 518 vocabulary rows would
exist but be unemittable, degrading tokenisation with no error. Embedding rows are copied
bit-exact — nothing is re-quantised.
Usage
lms load <model> --gpu max --context-length 8192 --parallel 1
Bonsai is a reasoning model. Budget ≥2500 output tokens, ideally 6000. With a
smaller budget the reply is consumed by chain-of-thought and content comes back empty —
measured at 75% of replies trapped with a 1200-token budget, 0% at 2500.
KV cache dominates VRAM once weights are this small — ~58 KiB/token across the 14
full-attention blocks. --parallel 4 quadruples it (896 MiB at 16K context against 3.0 GiB
of weights). Runs comfortably on an 8 GB card.
What did not work — the useful negative results
Perplexity does not track coding capability on this model. Four separate attempts moved
perplexity substantially and never moved pass@1 significantly:
attempt
perplexity
pass@1
norm-only healing (48 blk)
−5%
0/40
scale healing, hard targets (48 blk)
5.62 → 3.01 (−46%)
0/40, unchanged
KL distillation from Qwen3.5-35B-A3B (56 blk)
4.59 → 3.36 (−27%)
9.3% → 6.7%, p=0.40
KL distillation from Qwen3.5-35B-A3B (52 blk)
5.02 → 3.41 (−32%)
2.0% → 4.7%, p=0.20
The 48-block case is the starkest: healing beat the 3.803 GB parent on trajectory
perplexity (3.06 vs 4.51) while solving zero problems the parent solved seven of. It
learned to write more plausible wrong code — syntactically valid Python rose 70% → 80%
with pass@1 pinned at 0.
Mechanism: depth pruning removes computation; training the 186 M group scales adjusts
calibration. Scales are 0.8% of the model and can only rescale groups of 128 weights —
they can never change which direction a weight points. Distillation from a much stronger
teacher over an identical vocabulary did not bridge that, which is why this release is the
un-distilled model.
Valid-Python rate is not a proxy either: it stayed at 87.5% even at 2.5% pass@1.
Caveats
Not run through llama.cpp's own binary. Validated in LM Studio (which uses the
tokenizer embedded in the GGUF) and an independent PyTorch loader that reproduces
llama.cpp within ~3% on the parent.
Trimmed vocab drops CJK, Cyrillic and emoji — the measured corpus was English + code.
A trimmed model cannot be driven by the original 248,320-token tokenizer; that indexes
past the embedding and aborts with a CUDA device-side assert. Use the embedded one.
Long context untested — all perplexity used 2048-token windows, and pruning removed
2 of 16 full-attention blocks, which carry long-range attention.
n=150 on one benchmark measures a narrow slice of coding ability. No SWE-bench,
no agentic evaluation, no multi-turn testing.
Text-only; the parent's mmproj vision projector is not included.
License
Apache 2.0, inherited from prism-ml/Bonsai-27B-gguf. Base model by Prism ML, derived
from Qwen3.6-27B. All credit for the underlying model and its 1-bit quantisation belongs to
them; this repository contributes only depth pruning, vocabulary trimming, and measurement.
