DeepSeek-V4-Flash-0731-NVFP4-Q8attn-00001-of-00004.gguf through 00004: main model, 165.04 GB total
DeepSeek-V4-Flash-0731-DSpark-MXFP4-Q8attn.gguf: optional DSpark drafter, 10.90 GB

The 129 merged routed-expert tensors use NVFP4. Source FP8 attention/shared-expert linear tensors are Q8_0; embeddings and output head are BF16. The separate DSpark drafter uses nine MXFP4 expert tensors.

For the main model and fully GPU-resident DSpark drafter, 208 GiB or more of aggregate VRAM is recommended. The drafter also requires approximately 10.2 GiB of contiguous free VRAM on one GPU. Actual memory use depends on context length, batch size, and runtime settings.

Runtime requirement

This conversion needs the included llama.cpp-deepseek-v4-nvfp4-runtime.patch. Without it, DeepSeek V4 does not apply the NVFP4 per-expert scales and produces corrupted output. The patch applies to llama.cpp commit 5ecbe1ac17ec0484c5b44af0bd580cdc9c428ed4.

bash

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 5ecbe1ac17ec0484c5b44af0bd580cdc9c428ed4
git apply ../llama.cpp-deepseek-v4-nvfp4-runtime.patch

Main-model example:

bash

llama-server -m DeepSeek-V4-Flash-0731-NVFP4-Q8attn-00001-of-00004.gguf \
  -ngl all -c 32768 -fa on --jinja

Main model with the optional GPU-resident DSpark drafter:

bash

llama-server -m DeepSeek-V4-Flash-0731-NVFP4-Q8attn-00001-of-00004.gguf \
  -md DeepSeek-V4-Flash-0731-DSpark-MXFP4-Q8attn.gguf \
  --spec-type draft-dspark --spec-draft-n-max 5 \
  -ngl all -ngld all \
  -c 32768 -fa on --jinja

The main model and DSpark speculative decoding have both been runtime-validated. Performance and draft acceptance depend on hardware, prompt, context, and generation settings.

The GGUF metadata advertises a 1,048,576-token context. Longer contexts require additional KV-cache and compute memory.

axiomofmind/DeepSeek-V4-Flash-0731-NVFP4-GGUF

DeepSeek-V4-Flash-0731 NVFP4 GGUF

Files

Runtime requirement

axiomofmind/
DeepSeek-V4-Flash-0731-NVFP4-GGUF