These quants target the DS4 inference engine, not llama.cpp. https://github.com/antirez/ds4

They use DS4's deepseek4 tensor layout and metadata. Compatibility with llama.cpp, Ollama or LM Studio is untested — do not assume it works. Conversely, llama.cpp-format quants of this model (MXFP4 experts, BF16 tensors) cannot be loaded by ds4: those types are not in DS4's tensor type set.

Built from the official DeepSeek-V4-Flash-0731 release, which supersedes the V4 Flash preview.

Files

File	Size	Routed experts	Everything else
`DeepSeek-V4-Flash-0731-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-imatrix.gguf`	90.9 GiB	layers 0–36: `IQ2_XXS` (gate/up) + `Q2_K` (down); layers 37–42: `Q4_K` (all three)	`Q8_0` attn proj / shared experts / output, `F16` router + embed + indexer + compressor + HC, `F32` norms / sinks / bias

Sized for 128 GB Apple Silicon.

Quantization recipe

The filename is the spec.

Tensor class	Quant	Count
`blk.{0..36}.ffn_{gate,up}_exps`	`IQ2_XXS`	74
`blk.{0..36}.ffn_down_exps`	`Q2_K`	37
`blk.{37..42}.ffn_{gate,up,down}_exps`	`Q4_K`	18
`blk.*.attn_{q_a,q_b,kv,output_a,output_b}`	`Q8_0`	215
`blk.*.ffn_{gate,up,down}_shexp`	`Q8_0`	129
`output.weight`	`Q8_0`	1
`token_embd.weight`, `blk.*.ffn_gate_inp`	`F16`
`blk..attn_compressor_`, `indexer_`, `hc_`	`F16` / `F32`
`*_norm.weight`, `attn_sinks`, `exp_probs_b`	`F32`
`blk.{0,1,2}.ffn_gate_tid2eid`	`I32`	3

Routed experts are ~91% of the payload but each expert sees only a fraction of tokens, so aggressive quantization there costs less average quality than the same treatment of router, projections or shared experts. The last six layers keep Q4_K because late-layer error affects output disproportionately.

Imatrix provenance

Quantized with antirez's published importance matrix (DeepSeek-V4-Flash-chat-v2-routed-moe-ds4-1p5m.dat), which was collected on the V4 Flash preview, not on 0731.

The architecture, tokenizer and routing structure are byte-identical between the two releases, so the statistics apply dimensionally and should transfer well — an imatrix records which input columns carry activation energy, not the weight values themselves. But this has not been measured across the release boundary. A 0731-native imatrix may improve on it.

Notes on the 0731 release

0731 bundles the DSpark speculative-decoding module (mtp.0/1/2) into the same checkpoint, where the preview shipped it as a separate repository. A matching DSpark support GGUF can be built with deepseek4-quantize --dspark-support.

ds4's reasoning-effort prompt predates this release: --think max emits the prefix DeepSeek now labels high, and the new max tier has no corresponding string in the runtime. This affects prompt construction only, not the weights.

Usage

bash

git clone https://github.com/antirez/ds4 && cd ds4 && make

hf download schlaflos/DeepSeek-V4-Flash-0731-antirez-ds4-GGUF \
  DeepSeek-V4-Flash-0731-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-imatrix.gguf \
  --local-dir gguf

ln -sfn "$PWD/gguf/DeepSeek-V4-Flash-0731-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-imatrix.gguf" ds4flash.gguf

./ds4 -p "Explain Redis streams in one paragraph."

License

MIT. Base model copyright DeepSeek; GGUFs redistributed under the base model's release terms.