Note (this fork). This is not a new or re-trained model. The weights are
the official DeepSeek-V4-Flash-0731, byte-for-byte unchanged. The only
modification is setting num_experts_per_tok from 6 to 4 in config.json
(plus a one-line vLLM weight-loading shim to make it load). Everything else added
here is analysis: a top_k=4 vs top_k=6 accuracy/speed comparison and the
statistical evidence behind recommending top_k=4. See
Expert Routing: top_k=4 vs top_k=6.
Introduction
DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.
DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
Benchmark
DeepSeek-V4-Flash-0731
DeepSeek-V4-Flash (Preview)
DeepSeek-V4-Pro (Preview)
GLM-5.2
Opus-4.8
Terminal Bench 2.1
82.7
61.8
72.1
81.0
85.0
NL2Repo
54.2
39.4
38.5
48.9
69.7
Cybergym
76.7
38.7
52.7
-
83.1
DeepSWE
54.4
7.3
12.8
46.2
58.0
Toolathlon-Verified
70.3
49.7
55.9
59.9
76.2
Agents' Last Exam
25.2
15.8
16.5
23.8
25.7
AutomationBench Public
25.1
10.8
12.8
12.9
27.2
DSBench-FullStack †
68.7
37.0
41.8
61.8
71.6
DSBench-Hard †
59.6
25.8
31.1
54.5
71.7
Notes:
For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95.
† DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
Chat Template
This release does not include a Jinja-format chat template. Instead, we provide a dedicated encoding folder with Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model, and how to parse the model's text output. Please refer to the encoding folder for full documentation.
The reasoning_effort parameter now supports three levels — low, high, and max — which control how much deliberation the model spends before answering.
For example, the command below serves the model with vLLM on a single 4×GB300 node.
See the vLLM recipe for detailed instructions and other hardware configurations.
Expert Routing: top_k=4 vs top_k=6 (Recommended: top_k=4)
DeepSeek-V4-Flash-0731 ships with num_experts_per_tok=6 (6 of 256 routed experts
activated per token, ~13B active params). We additionally evaluated
num_experts_per_tok=4 (~11B active params) and recommend top_k=4 as the
default: it is measurably faster at statistically indistinguishable accuracy.
Why top_k=4
~15% fewer activated parameters, for free. Routing to 4 experts instead of
6 drops per-token active params from ~13B to ~11B. Shared experts and attention
are unchanged, so quality is preserved (see measurements below).
Faster inference (~13–18%). Fewer expert FFN computations plus less
gather/scatter and softmax overhead in the router. Measured end-to-end:
HumanEval wall-time ~15% lower, per-token generation ~13% faster.
Power-of-2 dispatch alignment.6 is not a power of two; MoE dispatch,
warp scheduling, and memory alignment on the GPU are more efficient when the
expert count aligns to a power of two (4), improving tensor-core utilization
for the dispatch/combine shapes.
No accuracy regression. On our internal SWE-bench-Lite and HumanEval runs
the difference between top_k=4 and top_k=6 is within run-to-run noise
(details below).
Single-GPU test environment (used for the numbers below)
The measurements were produced on a single NVIDIA B300 (SXM6, ~275 GiB) — no
data/expert parallelism and DSpark speculative decoding disabled (the official
multi-GPU launch in How to Run with vLLM enables DSpark;
that is orthogonal to the routing comparison here). Exact launch command:
No --speculative-config — DSpark is left off, so throughput numbers reflect
the base model. (DSpark is a decode-speed optimization and does not change which
tasks pass; it can be re-enabled independently.)
--max-model-len 32768 fits the KV cache comfortably in 275 GiB alongside the
fp8+MXFP4 weights; raise it if you have headroom.
The tid2eid shim from How to enable top_k=4 must be
applied before serving with num_experts_per_tok=4.
Weight load takes ~6–9 min (fp8/fp4 MoE autotuning on first start).
Test harnesses: HumanEval via /v1/chat/completions (code-fence extraction,
temperature=0.1); SWE-bench-Lite via the mini-swe-agent
minimal text-based agent (no Docker; each repo in an isolated uv venv), with the
official code-agent sampling temperature=1.0, top_p=0.95.
Measured accuracy — the difference is within noise
All numbers below are from the single-B300 setup above on the native
fp8+MXFP4 weights.
SWE-bench-Lite (n=86 subset, mini-swe-agent harness, no Docker,
official code-agent sampling temperature=1.0, top_p=0.95 unless noted):
Config
Resolved
Rate
Sampling
top_k=6
37/86
43.0%
default
top_k=4 (rep 1)
38/86
44.2%
default
top_k=4 (rep 2)
38/86
44.2%
default
top_k=4 (official)
39/86
45.3%
t=1.0, p=0.95
Two-proportion z-test top_k=4 vs top_k=6: z ≈ 0.15–0.31 (not significant).
Repeating the sametop_k=4 config flips ~14–17 of the 86 instances per pair of
runs (≈31% of the ever-solved union) purely from MoE-routing / fp8-kernel /
batching non-determinism. Aggregated over 4 runs: 23 instances always pass
(stable core), 37 always fail, and 26 are coin-flips. In other words, the
per-task differences between top_k=4 and top_k=6 are the same magnitude as
top_k=4 versus itself — i.e. noise, not a capability gap. top_k=4 gets the
speed and parameter savings at no measurable accuracy cost.
On knowledge-heavy multiple-choice benchmarks (e.g. MMLU-Pro) narrower routing
can even help slightly; on code generation top_k=6 may hold a fraction of a
point. Both directions are inside the noise band on our runs — treat them as
equivalent in quality.
How to enable top_k=4
Step 1 — set the config. In config.json:
"num_experts_per_tok": 4
Step 2 — patch vLLM weight loading (required). The checkpoint's tid2eid
tensor (the hash-based expert-routing lookup table) was trained at top_k=6, so
it has shape [vocab_size, 6]. With num_experts_per_tok=4 the model allocates a
[vocab_size, 4] parameter, and loading fails with:
AssertionError: Attempted to load weight (torch.Size([129280, 6]))
into parameter (torch.Size([129280, 4]))
Fix it by slicing the checkpoint tensor to the first top_k columns during load.
In vllm/models/deepseek_v4/nvidia/model.py, inside load_weights, in the
final else branch just before the weight_loader(param, loaded_weight) call:
python
1param = params_dict[name]2# top_k override: checkpoint's tid2eid is [vocab, 6] (trained at top_k=6);3# slice to the config's top_k columns so it matches the allocated parameter.4if"tid2eid"in name and loaded_weight.shape != param.shape:5 loaded_weight = loaded_weight[:,:param.shape[1]].contiguous()6weight_loader =getattr(param,"weight_loader", default_weight_loader)7weight_loader(param, loaded_weight)
The [:, :param.shape[1]] slice keeps the highest-priority expert columns and is
a no-op when the shapes already match (top_k=6), so the patch is safe to leave
in place for both configurations. No other weights change — total parameters,
routing method (noaux_tc), shared experts, and attention are all identical.
Note: this is a weight-loading shim, not a re-training of the router table.
The hash table's remaining 4 columns are the same top entries used at top_k=6,
which is why accuracy is preserved.
How to Run Locally
Please refer to the inference folder for detailed instructions on running DeepSeek-V4 locally, including model weight conversion and interactive chat demos.
For local deployment, we recommend setting the sampling parameters to temperature = 1.0, with top_p = 0.95 for agentic scenarios and top_p = 1.0 otherwise. For the high and max reasoning effort levels, we recommend a maximum output length of 384K tokens.
License
This repository and the model weights are licensed under the MIT License.
