Straight talk first: if you have a 512 GiB machine, you don't need this — the unpruned UD-IQ1_S (508 GB) runs fine there by streaming experts from SSD (llama.cpp MoE streaming, 5.3–6.3 tok/s decode measured on an M3 Ultra). This build exists for the machines below that: it cuts Qwen3.8 down to what an English + code deployment actually uses, along two axes at once —

expert pruning: keep the 304 of 512 experts per MoE layer that a measured en+code workload actually routes to;
width pruning: inside every surviving expert, keep the 6 of 8 256-channel superblocks that carry the most activation energy (intermediate width 2048 → 1536).

Result: 246 GB (229 GiB), quantization untouched — every surviving byte is byte-identical to the UD-IQ1_S it came from (1.56 bpw experts, no requantization anywhere). And no fork, no PR branch, no custom runtime: it loads and generates on stock mainline llama.cpp, verified.


experts	304 of 512 per MoE layer (uniform), selected by measured routing counts
expert width	1536 of 2048 (6 of 8 superblocks per expert), selected by measured activation energy
calibration	English web + code corpus (200k tokens) through the unpruned model: router selection counts (expert axis) + llama-imatrix activation stats (width axis)
selection coverage	89.6% of routing selections retained (worst layer 83.2%); kept width blocks carry 79.0% of activation energy (mean per expert)
quantization	untouched — byte-slab copy of UD-IQ1_S, no requantization
size	246 GB = 229 GiB
KLD vs unpruned UD-IQ1_S	held-out en: mean 0.239, median 0.097, argmax agreement 79.2%, PPL 9.05 → 10.90 (×1.20) · held-out code: mean 0.288, median 0.020, argmax agreement 86.6%, PPL 1.89 → 2.38 (×1.26)

Expert selection is count-based (how often the router picked each expert), not gate-weighted REAP saliency: on Kimi-K3 the same shortcut recovered 90.9% of saliency mass vs 93.5% for full saliency selection. Disclosed here because it is a methodological difference from our K3 builds. Width selection uses llama-imatrix's per-expert stats on the ffn_down_exps input — the squared activations of each intermediate channel — summed per 256-channel superblock.

Provenance: the exact pruning plan is published — plans/plan_ew_qwen_304x6.json (sha256 22aea5cfbc36dae5…, full hash in SHA256SUMS); plan + source quant + the MIT slicer reproduce this file's bytes. Split-half recalibration (same-distribution halves): keep-set overlap 92.7%, count-mass retention 98.7–98.9% (worst layer 97.0%) — details in plans/README.md.

Reading the quality numbers

How they were measured. ~128k tokens each of held-out English (FineWeb, skipped far past anything the calibration corpus touched) and held-out code (codeparrot validation) were teacher-forced through the unpruned UD-IQ1_S, saving its full next-token distributions; the pruned model then ran the identical text and llama-perplexity --kl-divergence compared the two, token by token. So every number below answers one question: how far does this build drift from the exact model it was cut from? (Not from FP16 — quantization loss is inherited from the parent and identical by construction.)

KL divergence — mean vs median. KLD is the per-token distance between the two probability distributions; 0 = identical. The shape matters more than the average: on code the median is 0.020 — half of all tokens are essentially untouched — while the mean of 0.288 is dragged up by a small heavy tail (99th percentile: 3.8) where the pruned model disagrees badly. Pruning damage is concentrated, not spread evenly. English shows the same shape but wider (median 0.097): the en+code calibration protects code harder than prose, by design.

Argmax agreement. The fraction of tokens where the pruned model's #1 choice equals the parent's — i.e., how often greedy decoding would pick the same token. 86.6% on code / 79.2% on English. For scale: a published K3 width-50 prune reported 73.7% against its parent; higher is better, and 100% would mean the prune changed nothing that greedy decoding can see.

Perplexity ratio. ×1.26 on code (1.89 → 2.38), ×1.20 on English. Treat this as a sanity check, not a verdict — PPL averages away exactly the tail structure that KLD exposes, and the tail is where agentic failures live.

Did the tail matter? Yes. The benchmark below is the tail made visible: routine tasks (head of the distribution) all pass; the five hardest tasks (which live in the tail) all fail. If your workload is routine coding assistance, these numbers say the prune barely touches you; if it leans on rare, hard reasoning, they say run the unpruned model.

Like every calibration-pruned build: what the corpus leaves out is what gets deleted. Non-English languages and off-domain abilities are deliberately sacrificed. Do not use this for multilingual work.

Verified vs. not verified

Verified:

claim	evidence
KLD / argmax agreement vs the unpruned quant	measured on held-out en and code text (table above)
Slicing is lossless for surviving weights	identity prune byte-identical; subset equals the naive numpy slice — pinned by tests in the tooling repo
Loads and generates	yes — on stock mainline llama.cpp (no fork needed; verified on `4c1a0af`) and on our fork; greedy outputs identical across both, 9.6–10.0 tok/s decode resident on an M3 Ultra
Drives an agent CLI	one-shot smoke with Qwen Code 0.21.11 against `llama-server --jinja`: wrote a file via tool calls, ran it, reported correct output. A smoke test, not a benchmark

Measured — SWE-Lancer (8 tasks, one attempt each, 10800 s cap, Qwen Code CLI 0.21.11):

3 of 8 tasks solved ($2,000 of the $13,500 at stake). The 8 tasks are real paid freelance jobs from the SWE-Lancer benchmark, in two groups:

3 easy sanity checks (tasks a 2-bit Kimi-K2.7 baseline could solve): all 3 passed — including one the unpruned parent model failed on a tool-calling format stumble.
5 hard tasks (selected precisely because that baseline failed them, prizes $500–$4,000): all 5 failed. Every failure was the model finishing with a wrong answer well under the 3-hour limit — not running out of time. The unpruned parent solves all five.

That is the KLD tail above, made concrete: routine agentic work survives the prune; the hardest tasks do not. If you have the RAM for the unpruned model, run that. Per-task results, exact conditions and every other build we compare against: swelancer-local-subset-evals.

Verified operating envelope — each number above was measured at a stated point: quality on 2048-token teacher-forced windows (the same regime the calibration ran in); all 8 agentic tasks served at context 131,072, sessions up to 163 min, including one task whose issue text alone is ~104k tokens. The parent model's trained window is 262,144. Beyond these points — longer contexts, non-English work (deliberately cut by calibration), off-corpus domains — you are past our measurements.

Run

Works with llama.cpp's Qwen3.5-MoE support (mainline or the fork we build):

bash

llama-server -m Qwen3.8-2.4T-A95B-REAP-256GB-IQ1_S.gguf \
    --port 8090 -ngl 99 -c 131072 --jinja \
    --temp 1.0 --top-p 0.95 --top-k 20

Sampling per the Unsloth card. The model is a hybrid thinking model (reasoning arrives in <think>; effort via reasoning_effort).

Memory reality check (macOS): 229 GiB is ~90% of a 256 GiB machine, and macOS caps Metal's working set well below that by default. Raise it (sudo sysctl iogpu.wired_limit_mb=<value>) and keep context moderate; if compute still hits the ceiling, lower -ngl a few layers — we hit exactly this on our 512 GiB machine with the 508 GB unpruned model (full offload OOMs at compute time; -ngl 80 runs). Honestly: 1-bit quants stream well — on a 256 GB machine, --moe-stream on the unpruned model is a perfectly good alternative to this build. This build is for when you want everything resident.

How it was made

Same pipeline family as our Kimi-K3 REAP builds (tooling, MIT), extended to two axes:

Expert axis: stream the calibration corpus through the unpruned model, count per-(layer, expert) router selections (via a small llama.cpp patch that dumps the MoE-streaming cache's hotness counters), keep the top 304 per layer.
Width axis: run llama-imatrix (resident) on the same corpus; its per-expert ffn_down_exps input stats are each intermediate channel's activation energy. Keep each expert's top 6 of 8 256-channel superblocks (scripts/make_plan_ew.py).
The MTP layer: blk.92 is a stored next-token-prediction (nextn) block that llama.cpp never executes — it has zero routing counts and no imatrix signal. It is sliced to the same uniform shape (required for the file to load) with an arbitrary expert selection; if some future runtime starts using MTP layers for speculative decoding, expect that path to be degraded in this build.
Slice (scripts/prune_gguf_ew.py): one pass over the GGUF, copying byte slabs. 256-channel cuts align with quant superblocks, so nothing is requantized — gate/up lose whole row blocks, down loses the matching intra-row spans, the router rows are renumbered to keep order, and expert_count / expert_feed_forward_length are rewritten. Pinned by synthetic identity/subset tests.

Credits: Qwen team (Qwen3.8), Unsloth (UD-IQ1_S quant this inherits), Cerebras REAP (the expert-pruning idea), mmnga (superblock-aligned width cuts), kimi-k3-mlx (the calibration methodology our pipeline grew from).

日本語の説明

Qwen3.8(2.4兆パラメータMoE)を、英語+コードの用途でほぼ使われない部分から二段階で削った版です。(1) 各層512個のexpertのうち実測ルーティングで選ばれない208個を削除、(2) 残ったexpertの中間層幅2048chのうち活性エネルギーの低い 512ch(256chブロック2個)を削除。量子化はいじらず全てバイトコピーなので、残った重みは元のUD-IQ1_Sと完全一致です。品質数値の読み方: KLDは「切り出し元とどれだけ分布がズレたか」(0=同一)。codeは中央値0.020=トークンの半分はほぼ無傷で、被害は上位1%のテールに集中(99%点3.8)。argmax一致86.6%は「greedyなら87%のトークンで親と同じ字を打つ」の意。詳細は英語節参照。246GB(229GiB)になり、256GiB機に全載せできます(Metalのwired上限引き上げ推奨、詳細は上のRunセクション)。512GiB機なら削っていない元をSSDストリーミングで動かす方が良い(実測5.3〜6.3 tok/s)。日本語を含む多言語は設計上壊れています。

hellohazime/Qwen3.8-2.4T-A95B-REAP-256GB-GGUF

Qwen3.8-2.4T-A95B, pruned to fit in 256 GB of memory.

Reading the quality numbers

Verified vs. not verified

Run

How it was made

日本語の説明

hellohazime/
Qwen3.8-2.4T-A95B-REAP-256GB-GGUF