One file that puts DeepSeek-V4-Flash (159B) on a single NVIDIA DGX Spark. This is
the offline-built 2-bit plane store for
vLLM-Moet: all 11,008 routed experts
(43 MoE layers × 256) of
deepseek-ai/DeepSeek-V4-Flash,
requantized to sign-symmetric
{−4,−1,1,4} 2-bit planes and serialized per-expert-slot
into one O_DIRECT-servable file. The serving stack reads experts straight from this file
into a GPU cache pool — the checkpoint's 148.66 GiB of expert tensors are
never
materialized in RAM, which is what makes a 119.7 GiB unified-memory box workable.
16.6 tok/s decode (29.8 with MTP k=2) · prefill ~413–665 tok/s · 64K window ·
greedy byte-identical 3/3 · needle 8/8 · tool + reasoning parsing on both the OpenAI
and Anthropic API surfaces. Full methodology and boot discipline:
docs/dgx-spark.md.
1# 1. this file + the upstream checkpoint
2hf download 9prodhi/DeepSeek-V4-Flash-moet2pf ds4-planes.moet2pf --local-dir /data/models
3hf download deepseek-ai/DeepSeek-V4-Flash \
4 --revision 60d8d70770c6776ff598c94bb586a859a38244f1 \
5 --local-dir /data/models/DeepSeek-V4-Flash
6
7# 2. serve (image build + all knobs: docs/dgx-spark.md in the GitHub repo)
8git clone https://github.com/9prodhi/vLLM-Moet && cd vLLM-Moet
9DOCKER_BUILDKIT=1 docker build -f Dockerfile.gb10-v024 -t vllm-moet-gb10:v024 .
10docker run -d --name vllm-moet --gpus all --network host --ipc host \
11 -v /data/models/DeepSeek-V4-Flash:/workspace/models/DeepSeek-V4-Flash:ro \
12 -v /data/models/ds4-planes.moet2pf:/workspace/models/ds4-planes.moet2pf:ro \
13 vllm-moet-gb10:v024
Boot only through the image's guarded entrypoint (it preflights the unified-memory
transient and refuses unsafe boots — a raw vllm serve can freeze a Spark).
Every slot carries a sha256; the serving stack re-verifies the file at each boot
(~12–15 s O_DIRECT). Offline:
1python3 tools/build_plane_file.py verify --dst /data/models/ds4-planes.moet2pf
2# -> OK: 11008 slots verified (a single flipped body byte fails loudly, no SIGBUS)
The file is deterministic output of
tools/build_plane_file.py
over the pinned checkpoint (~7 min on the Spark GPU, peak RSS < 1 GiB; slot bytes are
byte-identical to what the loader would stage in memory — checked by a golden sample at
build time). See
docs/dgx-spark.md §3, Option B.