Q4_K routed experts in layers 10, 30, and 39–42.
Other routed-expert layers use the Q2 imatrix recipe: IQ2_XXS gate/up and Q2_K down tensors.
Attention projections, shared experts, and output tensors follow the DS4 Q2-imatrix template.
Six of 43 layers use Q4 routed experts (13.95%).
File size: 97,591,747,456 bytes (90.89 GiB).

The selected Q4 tensors were copied byte-for-byte from the compatible Q4-imatrix GGUF. Other tensors remain byte-identical to the Q2-imatrix base.

Evaluation

All variants below were scored on the same 100 official DeepSeek V4 Flash continuations containing 2,289 target tokens.

Model / routed-expert recipe	Size	Average NLL ↓	NLL vs Q2	Wins vs Q2	First-token matches ↑	Average greedy LCP ↑
Q2 imatrix baseline	80.76 GiB	0.368177401	—	—	63 / 100	5.96
Antirez mixed recipe: Q4 layers `37–42`	90.89 GiB	0.340543836	-7.506%	67 / 33	68 / 100	6.90
This model: Q4 layers `10`, `30`, `39–42`	90.89 GiB	0.338330454	-8.107%	73 / 27	68 / 100	7.47
Full Q4 imatrix	153.33 GiB	0.173978006	-52.746%	87 / 13	86 / 100	12.88

Lower NLL is better. First-token matches count how often greedy generation starts with the same token as the official continuation. Greedy LCP is the number of consecutive matching tokens before the first divergence, so higher is better.

At the same 90.89 GiB size as the Antirez six-layer recipe, this configuration improved NLL, preserved the same first-token match count, and increased average greedy LCP from 6.90 to 7.47.

Usage

./ds4 \
  -m DeepSeek-V4-Flash-Layers10-30-39-42Q4KExperts-OtherExpertLayersQ2-imatrix.gguf \
  --ctx 4096 \
  -p "Hello"

This GGUF uses DS4-specific DeepSeek V4 tensor naming and mixed routed-expert quantization.

ivanfioravanti/deepseek-v4-gguf

DeepSeek V4 Flash — mixed Q2/Q4 GGUF

Quantization recipe

Evaluation

Usage

ivanfioravanti/
deepseek-v4-gguf