Stock DeepSeek-V4-Flash with DSpark speculative decoding. Faster than MTP on 2-4x RTX Pro 6000. By Fraser Price.
The official DeepSeek-V4-Flash (FP8 weights + FP4 experts, unmodified) with DeepSeek's DSpark draft module bolted on. DSpark (from DeepSpec) is block-wise speculative decoding: it drafts several tokens per step through a small Markov head, so single-stream decode is ~1.2-1.4x faster than the model's stock MTP at identical quality (speculative decoding is lossless). It runs on a Blackwell / CUDA-13.2 vLLM image that ships the dspark speculative method (voipmonitor/vllm:eldritch-enlightenment-v2226f26-b12x15cd38c-cu132-20260629), the image from local-inference-lab/rtx6kpro.
Update
The serving image has moved from the original fraserpricee/vllm:dspark-cu132-20260627 to the local-inference-lab vLLM build (voipmonitor/vllm:eldritch-enlightenment-v2226f26-b12x15cd38c-cu132-20260629) — a more performant DSpark + sparse-MLA implementation on Blackwell. All credits to them!
The performance numbers below were measured on this V2 build, weights are the same.
That one command grabs the launcher, downloads the weights (~167 GB) into your Hugging Face cache, pulls the inference image, and serves an OpenAI-compatible API on http://localhost:8000/v1. No other files or arguments needed. On 4 GPUs:
GPUS=0,1,2,3 TP=4 bash run.sh
Everything is overridable by env var:
Variable
Default
Meaning
GPUS
0,1
GPU indices (comma-separated)
TP
2
Tensor-parallel size; set to the number of GPUs
DSPARK_TOKENS
5
DSpark draft tokens per step
PORT
8000
API port
MAX_MODEL_LEN
262144
Context length
GPU_MEM_UTIL
0.92
Fraction of VRAM to use
HF_REPO
fraserprice/DeepSeek-V4-Flash-DSpark
repo to download
MODEL_DIR
(HF cache)
serve a local dir instead of downloading
SERVED_NAME
DeepSeek-V4-Flash-DSpark
API model name
IMAGE
(pinned DSpark image)
inference container
Hardware
Tuned for NVIDIA RTX Pro 6000 Blackwell (96 GB, sm_120). Needs roughly 180 GB total VRAM for weights + DSpark module + KV cache at long context, split across GPUs by tensor parallelism.
Setup
Works?
2 x RTX Pro 6000 (96 GB), TP=2
✅ reference config
4 x RTX Pro 6000, TP=4
✅ more KV headroom / throughput
< ~180 GB total VRAM
❌ won't fit
The serving image targets Blackwell (sm_120) / CUDA 13.2 and adds the dspark speculative method to vLLM. See local-inference-lab/rtx6kpro for the image source.
Performance
Measured on RTX Pro 6000 Blackwell (PCIe 5.0), FP8 weights + FP8 KV cache, gpu_memory_utilization=0.92, DSpark num_speculative_tokens=5, 256 generated tokens/request. PP is prefill throughput (tok/s); TG is per-request decode throughput (tok/s); total is aggregate throughput (tok/s) across concurrent requests. MTP TG is the base model's shipped single-layer-MTP decode for reference; Decode ↑ is DSpark's gain over it.
These numbers were measured on the abliterated DSpark twin and apply identically here: same architecture, precision, and DSpark module. Abliteration is a rank-3 projection on the base weights that does not change compute.
Latency
GPUs
prompt
conc
TTFT p50
ITL p50
E2E p50
2
1,000
1
155 ms
15.6 ms
1.34 s
2
1,000
3
435 ms
23.1 ms
2.04 s
2
10,000
1
1,232 ms
15.9 ms
2.32 s
2
10,000
3
2,495 ms
23.8 ms
5.30 s
2
100,000
1
14,568 ms
16.4 ms
15.71 s
2
100,000
3
29,078 ms
26.1 ms
44.77 s
4
1,000
1
127 ms
11.9 ms
0.96 s
4
1,000
3
259 ms
16.3 ms
1.52 s
4
10,000
1
983 ms
12.0 ms
1.87 s
4
10,000
3
2,229 ms
16.9 ms
4.06 s
4
100,000
1
12,772 ms
12.5 ms
13.59 s
4
100,000
3
26,698 ms
18.9 ms
40.11 s
Throughput (tok/s)
GPUs
prompt
conc
PP
MTP TG
DSpark TG
Decode ↑
total
2
1,000
1
8,080
190.9
224.7
1.18×
1,160
2
1,000
3
8,377
138.4
152.5
1.10×
2,120
2
10,000
1
10,081
191.5
239.5
1.25×
5,469
2
10,000
3
10,050
90.3
106.6
1.18×
7,046
2
100,000
1
8,525
185.4
247.1
1.33×
7,965
2
100,000
3
8,585
60.4
67.1
1.11×
8,320
4
1,000
1
9,769
233.3
322.7
1.38×
1,605
4
1,000
3
10,578
181.9
212.8
1.17×
2,927
4
10,000
1
12,645
234.5
314.3
1.34×
6,984
4
10,000
3
12,642
115.0
148.3
1.29×
9,122
4
100,000
1
9,734
232.8
335.8
1.44×
9,194
4
100,000
3
9,507
74.7
92.7
1.24×
9,278
~1.2-1.4x faster single-stream decode than the base model's shipped MTP, at the same (lossless) quality.
local-inference-lab/rtx6kpro: the sm120 Blackwell / CUDA-13.2 vLLM build (voipmonitor/vllm:eldritch-…) that serves this model, with the dspark speculative method and B12X sparse-MLA stack.
voipmonitor/vllm: the vLLM image registry the image is published to.
License
MIT, inheriting from the base model.
Built by Fraser Price, @fraserpricee. Found this useful? A follow is appreciated.
