This model has MTP (Multi-Token Prediction) weights in the form of a DSpark draft model. Unlike other MTP implementations, these are not included in the quants themselves - they are provided as a separate file in this repo: dspark-DeepSeek-V4-Flash-0731-MXFP4.gguf
DSpark acts as a draft model, letting llama.cpp run speculative decoding for faster generation. To use it, add the following flag to your llama.cpp command:
--spec-type draft-dspark
When running with -hf as shown above, llama.cpp downloads the DSpark file automatically alongside the model. If you're downloading files manually instead, also grab the dspark file and pass it with -md /path/to/dspark-DeepSeek-V4-Flash-0731-MXFP4.gguf.
Note: the DSpark file keeps the original model's MXFP4 expert weights unchanged (remaining tensors at Q8_0 or higher).
Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
