This is not a conventional standalone GGUF. The non-routed and shared tensors are stored in model-dense.gguf; routed expert tensors are stored separately in a layer-major sidecar/ and loaded into a per-layer resident slot bank on demand. Use the Flash-MoE-enabled Qwen3.8 branch of anemll-flash-llama.cpp. Stock llama.cpp builds that do not understand this package layout or its IQ1_XXXS routed experts cannot run it.

Model and package details

Property	Value
Architecture	`qwen35moe`
Transformer blocks	93
Embedding width	8192
Context metadata	262,144 tokens
Routed experts per layer	512
Native experts selected per token	10
Dense/shared GGUF tensors	1,423
Dense/shared GGUF file size	36,881,771,680 bytes
Routed tensors	279
Routed sidecar size	360,374,599,680 bytes
Complete upload size	397,256,650,449 bytes (369.97 GiB)

The routed sidecar preserves the source GGUF bytes exactly:

Layers 0-91: gate, up, and down routed tensors are IQ1_XXXS (276 tensors).
Layer 92: gate, up, and down routed tensors are Q2_K (3 tensors).
Each layer file contains the three whole routed tensors for that layer, with 512 experts packed within each tensor.

The dense/shared GGUF quant inventory is:

Quant type	Tensors	Tensor-data bytes
`F32`	838	2,171,133,440
`Q4_K`	2	2,288,517,120
`Q5_K`	420	23,391,633,408
`Q6_K`	162	8,876,851,200
`Q8_0`	1	142,606,336

Download

The repository is about 397 GB. Put it on a fast local SSD and preserve the directory layout:

bash

hf download anemll/Qwen3.8-2.4T-A95B-FlashMoE-UD-Q1_0 \
  --local-dir ~/Models/Qwen3.8

Expected layout:

text

~/Models/Qwen3.8/
├── model-dense.gguf
├── flashmoe-package.json
└── sidecar/
    ├── manifest.json
    ├── layer_000.bin
    ├── ...
    └── layer_092.bin

Keep the sidecar on fast directly attached storage. This runtime uses SSD reads for cache misses; network filesystems and slow external links will sharply reduce decode speed.

SSD slot-bank inference

Build the Flash-MoE branch of anemll-flash-llama.cpp, then run from its source directory:

bash

./build/bin/llama-cli \
  -m ~/Models/Qwen3.8/model-dense.gguf \
  --moe-mode slot-bank \
  --moe-sidecar ~/Models/Qwen3.8/sidecar \
  --moe-slot-bank 96 \
  --moe-topk 10 \
  --moe-cache-io-split 4 \
  --moe-prefetch-temporal \
  -fit on \
  -ub 1 \
  -b 1 \
  -ngl 999 \
  -c 128 \
  --no-warmup \
  -st \
  -p "Hello" \
  -n 16

Important runtime choices:

Keep --moe-topk 10: this is the model's native routed width.
Do not add --slot4 or --slot8; those fused paths do not match native top-10 routing.
-ub 1 and -b 1 select the decode-oriented SSD path.
--moe-prefetch-temporal overlaps next-layer reads with current-layer compute.
The 96-slot setting is intended for a 128 GB Apple Silicon machine. Each slot, summed over all 93 layer banks, costs 703,856,640 bytes (671.25 MiB), so a 96-slot bank uses about 62.93 GiB in addition to the dense model and runtime allocations. If memory pressure is high, start with 32 or 64 slots.
The 262K value is model metadata, not a promise that the full context fits in memory. Increase -c only after budgeting KV-cache and runtime memory.

Attribution and license

Base model: Qwen/Qwen3.8-2.4T-A95B
Source quantization: unsloth/Qwen3.8-2.4T-A95B-GGUF, UD-Q1_0
Flash-MoE packaging/runtime: ANEMLL anemll-flash-llama.cpp, Qwen3.8 branch

The embedded GGUF metadata reports license: other. Review and comply with the base model and source repository license terms before use or redistribution.

Verification metadata

flashmoe-package.json records the export summary and sidecar/manifest.json records source offsets, quant types, shapes, exact byte lengths, and repacked offsets. The package was produced with byte-level sidecar verification enabled.