This repository contains only the practical bullerwins Pareto points retained after size/KLD testing. Models that were dominated or offered only negligible size advantages over a better Unsloth quant were removed.
The routed MoE experts in the lossless reference remain in their native MXFP4 representation. FP8-origin non-expert matrices are dequantized into BF16, which represents their E4M3 values and power-of-two E8M0 scales exactly. Original BF16 and F32 tensors are also preserved. The resulting MXFP4_MOE-BF16 file is lossless relative to the model tensors used by llama.cpp.
MXFP4 down; IQ2_XS gate/up in layers 0-17; IQ3_XXS gate/up in layers 18-42
Eligible non-expert matrices in Q8_0
120.33 GB / 112.07 GiB
DeepSeek-V4-Flash-0731-IQ3_XXS-Experts-Q8_0.gguf
IQ3_XXS
Eligible non-expert matrices in Q8_0
113.87 GB / 106.05 GiB
DeepSeek-V4-Flash-0731-IQ2_XS-Experts-Q8_0.gguf
IQ2_XS
Eligible non-expert matrices in Q8_0
87.90 GB / 81.86 GiB
DeepSeek-V4-Flash-0731-DSpark.gguf
DSpark speculative companion
Mixed MXFP4/Q8_0/BF16/F32
10.90 GB / 10.15 GiB
DeepSeek-V4-Flash-0731-imatrix.gguf
-
Importance matrix generated from 802 x 512-token calibration chunks
0.47 GB / 0.44 GiB
The IQ quantizations were generated with the included calibration importance matrix. The mixed autoresearch model keeps all 43 expert down projections in native MXFP4, uses IQ2_XS for the 36 gate/up tensors in layers 0-17, and uses IQ3_XXS for the 50 gate/up tensors in layers 18-42.
KLD quality evaluation
All models were tested against logits from the lossless MXFP4_MOE-BF16 reference using 50 x 512-token Wikitext-2 test chunks. The test evaluated 12,750 output distributions. Lower KLD and higher same-top-token agreement are better. Autoresearch policies were selected on the separate Wikitext-2 validation split; the test split was used only as a holdout check.
Model
Mean KLD
PPL ratio
Same top token
Lossless MXFP4/BF16
0.000000
1.0076
100.000%
Mixed IQ2_XS/IQ3_XXS gate/up + MXFP4 down
0.280219
1.1379
84.094%
IQ3_XXS experts/Q8_0
0.310419
1.1604
83.067%
IQ2_XS experts/Q8_0
0.600273
1.4658
75.082%
Comparison with Unsloth Dynamic quants
KLD versus GGUF size for the retained bullerwins Pareto set and Unsloth
Both collections provide strong and closely competitive quality/size tradeoffs. The lossless bullerwins and Unsloth models have byte-identical tensor payloads. The bullerwins IQ2_XS model fills the gap between Unsloth's IQ1 and IQ2 tiers, while bullerwins IQ3_XXS and the 112.07 GiB mixed model provide useful intermediate points between Unsloth UD-IQ3_XXS and UD-Q3_K_M. Unsloth remains stronger around 90, 119, 127, and 144 GiB and also offers smaller sub-80 GiB options. The plot intentionally shows only the bullerwins models retained as practical Pareto points.
Source model
DeepSeek's recommended sampling settings are temperature = 1.0, with top_p = 0.95 for agentic scenarios and top_p = 1.0 otherwise. See the official model repository for architecture details, chat formatting, license, and intended usage.
DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.
DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
Benchmark
DeepSeek-V4-Flash-0731
DeepSeek-V4-Flash (Preview)
DeepSeek-V4-Pro (Preview)
GLM-5.2
Opus-4.8
Terminal Bench 2.1
82.7
61.8
72.1
81.0
85.0
NL2Repo
54.2
39.4
38.5
48.9
69.7
Cybergym
76.7
38.7
52.7
-
83.1
DeepSWE
54.4
7.3
12.8
46.2
58.0
Toolathlon-Verified
70.3
49.7
55.9
59.9
76.2
Agents' Last Exam
25.2
15.8
16.5
23.8
25.7
AutomationBench Public
25.1
10.8
12.8
12.9
27.2
DSBench-FullStack †
68.7
37.0
41.8
61.8
71.6
DSBench-Hard †
59.6
25.8
31.1
54.5
71.7
Notes:
For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95.
† DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
Chat Template
This release does not include a Jinja-format chat template. Instead, we provide a dedicated encoding folder with Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model, and how to parse the model's text output. Please refer to the encoding folder for full documentation.
The reasoning_effort parameter now supports three levels — low, high, and max — which control how much deliberation the model spends before answering.
For example, the command below serves the model with vLLM on a single 4×GB300 node.
See the vLLM recipe for detailed instructions and other hardware configurations.
Enable DSpark with --speculative-algorithm DSPARK and do not set a separate --speculative-draft-model-path as the target and draft weights therefore come from the same checkpoint.
See the SGLang cookbook for detailed instructions, benchmarks and other hardwares configurations.
Please refer to the inference folder for detailed instructions on running DeepSeek-V4 locally, including model weight conversion and interactive chat demos.
For local deployment, we recommend setting the sampling parameters to temperature = 1.0, with top_p = 0.95 for agentic scenarios and top_p = 1.0 otherwise. For the high and max reasoning effort levels, we recommend a maximum output length of 384K tokens.
License
This repository and the model weights are licensed under the MIT License.
