Qwen3.6-35B-A3B pruned to 19B total / ~3.3B active parameters, stored at 2-bit — 6.8× less expert memory than the 8-bit reference. Loads with the 30-line shim shipped in this repo; everything else is stock mlx_lm.

Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts.

A paper with the full method and measurements is coming soon.

Benchmarks

Full evaluation protocol: WikiText-2 (145 × 2048-token windows) and C4 (256 × 2048-token windows) perplexity; task average over arc_easy, hellaswag, piqa, winogrande, and boolq, paired against the reference.

model	expert storage	wt2 ppl ↓	task avg ↑	retention	generation d2 (avg/min)
reference (8-bit)	34.2 GB	6.76	0.790	100%	0.63 / 0.51
Qwen3.6-27B-A3B-Niwaki-2bit-mlx	7.6 GB (0.221×)	9.89	0.739	93.5%	—
Qwen3.6-19B-A3B-Niwaki-2bit-mlx	4.7 GB (0.147×)	12.38	0.712	90.1%	—
this model	4.7 GB (0.147×)	10.82	0.712	90.1%	0.84 / 0.54

Same size and same bytes as Qwen3.6-19B-A3B-Niwaki-2bit-mlx, 12.6% lower perplexity, equal task average. Generation quality is measured, not assumed: an 8-prompt battery (code, reasoning, chat, creative; 600-token sampled generations) scored by bigram diversity, reference measured under the identical battery.

Evaluation methodology

Task average: five benchmarks from lm-evaluation-harness scored by per-choice log-likelihood — arc_easy, hellaswag, and piqa report normalized accuracy (acc_norm); winogrande and boolq report accuracy (acc); the average is the unweighted mean of the five. Every row is evaluated on the same 500 samples per task (paired screens), so the numbers are directly comparable down the column. Perplexity rows use the full window protocol above, identical windows for every row, loss in fp32, no chat template.

Model dimensions


total parameters	~19B (from 35B)
active parameters per token	~3.3B (from ~3.8B)
layers / routed experts / top-k	40 / 256 / 8 (layers 10–29 are shared-expert-only)
expert intermediate size	512 (unchanged)
expert precision	2-bit (group 64)
backbone precision	8-bit (untouched)
whole-model size on disk	~7.6 GB
runs comfortably in	16 GB unified memory

Usage (MLX, Apple Silicon)

This checkpoint stores no tensors for the removed expert banks, which stock mlx-lm cannot express — load through the shim in this repo (it stubs the routed path of the shared-only layers, then defers to mlx_lm.load):

python

# curl -O https://huggingface.co/neopolita/Qwen3.6-19B-A3B-Niwaki-v2-2bit-mlx/resolve/main/niwaki_v2_load.py
from niwaki_v2_load import load
from mlx_lm import generate

model, tokenizer = load("neopolita/Qwen3.6-19B-A3B-Niwaki-v2-2bit-mlx")
print(generate(model, tokenizer, prompt="...", max_tokens=256))

GGUF builds for llama.cpp: Qwen3.6-19B-A3B-Niwaki-v2-2bit-gguf.

Notes and limitations

Compression trades quality: this model retains ~90% of the reference's task performance. Choose the family member that fits your memory budget.
Evaluated text-only on English-web-heavy data; the base model's biases are inherited and rare-domain behaviour is less tested.
No latency claim: all 40 layers execute. This is a memory artifact.
Part of the Niwaki family: Qwen3.6-19B-A3B-Niwaki-v2-4bit-mlx (quality tier), and the first-generation 27B / 19B / 11B models.

Base model by the Qwen team (Apache 2.0); 8-bit MLX conversion by mlx-community; compression and distillation by the Niwaki project, 2026-08.