Full speculative-decoding benchmark, target model DeepSeek-V4-Flash-0731-UD-IQ3_XXS-q6kattn.gguf (96 GB, IQ3_XXS + Q6 attention), 7 task categories × 6 draft lengths × 5 runs each (210 requests per draft model). Sampling: temp 0.9 / top_p 0.95 / min_p 0.01, ctx 65536, reasoning_effort=low.

Baseline (no draft model): 20.48 tok/s.

Headline numbers

Metric	Q8_0 draft	Q2_K_S draft (this)
File size	10.15 GB	6.45 GB (−36%)
Best overall decode	28.43 tok/s	28.58 tok/s
Best overall speedup vs baseline	1.39×	1.40×
Optimal n_max (overall)	3	3

Q2_K_S matches the Q8_0 drafter within noise (±1–3% on every n_max and every category, tie per the benchmark's own threshold) while being 36% smaller than Q8_0 and 39% smaller than the BF16 drafter (10.54 GB).

Speedup vs baseline (20.48 tok/s) by task category

Best result per category, averaged over Q8_0 + Q2_K_S drafts:

Category	Best n_max	Decode tok/s	Speedup
code	3	28.99	1.42×
prose	2	23.38	1.14×
translate	2	25.09	1.22×
json	3	28.59	1.40×
math	3	31.22	1.52×
repeat	5	40.17	1.96×
dialog	3	25.48	1.24×
overall	3	28.50	1.39×

Note on the averages. The 28.5 tok/s overall figure is pulled up by the mix: the suite is heavy on tasks that speculative decoding likes (repeat, math, code). Prompts also used a relatively short context (up to ~32k tokens of the 64k window). On a more typical mixed workload — especially with a fuller context — expect closer to 22–28 tok/s.

n_max sweep (average of Q8_0 + Q2_K_S)

n_max	Decode tok/s	Speedup	Accept rate
2	27.12	1.32×	0.693
3	28.50	1.39×	0.602
4	27.73	1.35×	0.522
5	26.43	1.29×	0.455
6	26.40	1.29×	0.456
7	26.52	1.29×	0.451

n_max=3 is the sweet spot: longer drafts (n_max≥5) buy almost nothing and even start to hurt on prose/translate/dialog (low acceptance → wasted draft tokens).

Practical recommendation

Run the server with --spec-draft-n-max 3. If the workload is code/json/math-heavy, n_max=3 is optimal; for repeat-like (high-acceptance) loads, n_max=5–6 adds ~1.5× more. The Q2_K_S drafter is the drop-in choice: same speed as Q8_0 at 36% smaller footprint. Treat the 28.5 tok/s headline as an optimistic mix, not a day-to-day guarantee; 22–28 tok/s is the realistic band.

Charts

Full chart set in bench_reports/20260817_225557/charts/:

Decode tok/s by category — average of Q8+Q2

Also in the same directory: per-draft decode charts (decode_tps_by_category_q8.png, _q2.png, overlay variant), per-draft heatmaps (speedup_heatmap_q8.png, _q2.png), decode_tps_vs_nmax_avg.png, speedup_grouped_bars.png, speedup_vs_accept_rate.png.

Benchmark setup

Baseline 20.48 tok/s measured without -md / --spec-type on the same target model
Categories: code, prose, translate, json, math, repeat, dialog
Server: llama-server from strix-halo-llamacpp, -fa 1, -ngld 999, -np 1, draft-dspark, ngram-mod off
5 runs per cell; reported values are averages with min–max spread

Quantization Details

Quantized using upstream llama.cpp llama-quantize with --allow-requantize and custom tensor-type overrides.

Expert weights (92% of model):

gate/up/down_exps: Q2_K_S

Attention & shared experts:

attn_kv, attn_output, attn_q_b: Q8_0
attn_q_a: Q5_K
ffn_*_shexp: Q5_K/Q6_K
fc: Q6_K

Protected:

MoE router, markov weights, hyper connections: F16/Q8_0
All norms, sinks, biases: F32

Credits

Base model: DeepSeek-V4-Flash-0731
BF16 GGUF: unsloth/DeepSeek-V4-Flash-0731-GGUF
Quantization: llama.cpp (upstream)
Inference & benchmarking: strix-halo-llamacpp

Lynxpda/DeepSeek-V4-Flash-0731-DSpark-Drafter-Q2_K_S-GGUF

DeepSeek-V4-Flash-0731 DSpark Drafter Q2_K_S

Performance (AMD Strix Halo)