Qwen3.8 REAP30 Width50 UD GGUF
An experimental reduced GGUF build of
Qwen3.8-2.4T-A95B.
The routed-expert count is reduced by about 30%, then the intermediate width of each retained expert is reduced by 50%. The number of experts selected per token remains 10.
Calibration and selection
The calibration set contains 97 completed English and Japanese samples covering general chat and reasoning, code generation, and tool use. It contains 94,874 profiled tokens, of which 78,061 are semantic assistant tokens used for scoring. Prompt and structure tokens are excluded.
REAP30: expert axis
For expert e, the score is:
1REAP(e) = mean[t routed to e](
2 top-10-renormalized router weight(t,e)
3 × ||unweighted expert output(t,e)||₂
4)
Experts are ranked independently in each of the 92 MoE layers. The top 358 of 512 are retained. No experts are reserved manually.
Width50: intermediate axis
After the REAP30 slice, the reduced model is profiled again with the same calibration set. Every intermediate channel is scored independently for each expert:
1a(t,e,c) = SiLU(gate(t,e,c)) × up(t,e,c)
2score(e,c) = Σ[t routed to e](router_weight(t,e) × a(t,e,c))²
3 × ||Wdown,e[:,c]||₂²
The top 1,024 of 2,048 channels are retained for each expert. Low-coverage experts use a blend of the activation score and a static weight-energy prior.
GGUF changes
| Original | This build |
|---|
| MoE layers | 92 | 92 |
| Routed experts per layer | 512 | 358 |
| Routed expert FFN width | 2,048 | 1,024 |
| Experts selected per token | 10 | 10 |
The router and grouped expert tensors are sliced with the same per-layer expert map. Shared experts are retained. MTP tensors are omitted.
Width selection is channel-level rather than 256-channel-block-level. Retained gate/up rows are copied directly. ffn_down_exps.weight is dequantized, gathered by the selected channel map, packed into new QK256 blocks, and requantized to the source GGUF type.
Both folders use the same REAP and Width rankings.
Files
| Folder | Shards | Size |
|---|
UD-IQ1_S | 12 | about 184.72 GiB |
UD-Q1_0 | 10 | about 148.54 GiB |
Load the first shard in the selected folder. Each folder also includes slice_manifest.json with the applied ranking hashes and slice settings.
Usage
UD-Q1_0 uses the experimental IQ1_XXXS GGML type. Use the
iq1-narrow branch:
1git clone --branch iq1-narrow https://github.com/unslothai/llama.cpp
2cmake -S llama.cpp -B llama.cpp/build \
3 -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
4cmake --build llama.cpp/build -j --target llama-server
Example for a 32 GB GPU while keeping MoE weights on CPU:
1./llama.cpp/build/bin/llama-server \
2 -m ./UD-Q1_0/Qwen3.8-2.4T-A95B-UD-Q1_0-00001-of-00010.gguf \
3 --cpu-moe \
4 -ngl 80 \
5 --ctx-size 8192 \
6 --flash-attn on \
7 --cache-type-k q8_0 \
8 --cache-type-v q8_0
This configuration was tested with 512 GB system RAM and one 32 GB RTX 5090. It offloaded 80 of 92 layers and used about 30.6 GiB of VRAM. Adjust -ngl and context size for your hardware.
This is a heavily reduced experimental model. Quality and stability may differ from the original model and depend on how closely the calibration distribution matches the target workload.
