This repository does not contain a standalone language model. The GGUF file is a draft/speculator model and must be loaded together with a compatible Gemma 4 31B-it target model.

If your target has a native MTP head available, use that instead. On this target and hardware, native MTP remains substantially faster and more consistent than either DSpark or DFlash. DSpark is mainly useful for targets without a compatible native MTP head, or for testing external speculator support.

Status

Support for DSpark-format speculators is being added to llama.cpp in ggml-org/llama.cpp#26275.

The latest benchmark below was re-run after the updated Speculators / DSpark handling in PR #26275. Compared with the previous result, DSpark throughput improved noticeably while its acceptance statistics remained effectively unchanged.

Until PR #26275 is merged, use a build compiled from the PR branch.

Compatibility

Component	Requirement
Target architecture	Gemma 4 31B
Recommended target	`google/gemma-4-31B-it` or a compatible GGUF quantization
Draft format	DSpark
Runtime	`llama.cpp` with PR #26275 or later merged support
Standalone use	No

The target model and speculator must use compatible tokenizer, vocabulary and architecture metadata. A mismatch can cause loading errors or incorrect output.

Recommended settings

text

speculative type: DSpark
draft max: 3

max 3 remains the best mixed-workload setting.

max 2 has the highest acceptance ratio but leaves some useful speculation depth unused. max 4 is still competitive, but aggregate throughput begins to decline. max 5 clearly over-drafts and loses performance.

text

llama-cli --help
llama-server --help

Command-line option names may change while PR #26275 is under review — check the help text of the build you are using.

Benchmark

Setup:

Target: gemma-4-31B-it-UD-Q4_K_XL
GPUs: RTX 5070 Ti 16GB + RTX 5060 Ti 16GB
13 requests
Single-stream mixed workload
Same benchmark workload used for DSpark / DFlash comparison

Mode	Predicted	Draft	Accepted	Acceptance	Effective rate	vs. baseline
No speculation	1,990	0	0	—	26.16 tok/s	1.00x
DSpark max 5	1,984	2,943	1,373	46.65%	40.51 tok/s	1.55x
DSpark max 4	1,984	2,469	1,345	54.48%	43.64 tok/s	1.67x
DSpark max 3	1,982	2,085	1,268	60.82%	44.47 tok/s	1.70x
DSpark max 2	1,982	1,610	1,157	71.86%	44.17 tok/s	1.69x
DFlash max 6	1,984	3,050	1,461	47.90%	43.07 tok/s	1.65x
Native MTP max 2	1,982	1,488	1,223	82.19%	55.67 tok/s	2.13x

Change from the previous DSpark result

The previous conversion/runtime path produced:

Mode	Previous	Updated	Change
DSpark max 2	39.37 tok/s	44.17 tok/s	+12.2%
DSpark max 3	40.77 tok/s	44.47 tok/s	+9.1%
DSpark max 4	39.95 tok/s	43.64 tok/s	+9.2%
DSpark max 5	37.24 tok/s	40.51 tok/s	+8.8%
DFlash max 6	43.10 tok/s	43.07 tok/s	essentially unchanged

The especially interesting part is that DSpark's draft and acceptance statistics remained effectively unchanged while throughput increased.

For example, at max 3:

text

Predicted:  1,982
Draft:      2,085
Accepted:   1,268
Acceptance: 60.82%

Those values are unchanged from the previous run, yet effective throughput increased from 40.77 → 44.47 tok/s.

DFlash, meanwhile, stayed almost exactly where it was: 43.10 → 43.07 tok/s.

This strongly suggests that the improvement is primarily in the updated DSpark conversion/runtime path rather than improved speculative prediction quality or a general DFlash/speculative-decoding speedup.

Per-workload (tok/s)

Workload	No spec	DSpark max 3	DSpark max 4	DFlash max 6	Native MTP max 2
Python code	27.2	47.5	52.8	57.6	64.7
C++ code	27.2	50.5	52.0	48.5	60.5
Concept explanation	27.4	40.2	33.8	30.4	53.5
Summarization	27.7	50.5	44.3	40.1	57.2
Factual QA	27.4	44.5	39.8	35.9	54.0
Translation	28.6	19.4	17.2	21.4	59.8
Creative writing	27.9	39.3	36.3	24.4	49.2
Stepwise math	27.4	57.5	59.7	56.2	66.0
JSON output	27.3	59.0	59.7	68.9	68.5
Long reasoning	27.3	52.6	51.0	57.1	65.4
Repeated pattern	27.4	63.2	71.6	83.2	68.7
Code completion	27.2	55.7	59.1	68.3	66.2
Long code review	26.4	37.8	33.1	30.0	52.5

Interpretation

Gemma 4 31B is a dense target, so each verified target token is relatively expensive. This gives speculative decoding substantial target computation to save.

With the updated DSpark path, max 3 takes the baseline from:

26.16 → 44.47 tok/s

or about:

1.70x baseline throughput

DSpark now slightly beats DFlash overall

The previous benchmark showed DFlash ahead:

text

DSpark max 3: 40.77 tok/s
DFlash max 6: 43.10 tok/s

After the updated DSpark conversion/runtime changes:

text

DSpark max 3: 44.47 tok/s
DFlash max 6: 43.07 tok/s

DSpark is now approximately 3.3% faster than DFlash on aggregate throughput in this mixed workload.

The margin is small enough that it should not be interpreted as a universal architectural win, but it reverses the previous result on this hardware and workload.

DSpark and DFlash have different workload strengths

DFlash remains particularly strong when output is highly predictable or structurally constrained:

JSON output
repeated patterns
code completion
Python code
long reasoning

For example:

text

Repeated pattern:
DSpark max 3  63.2 tok/s
DFlash max 6  83.2 tok/s

JSON:
DSpark max 3  59.0 tok/s
DFlash max 6  68.9 tok/s

Code completion:
DSpark max 3  55.7 tok/s
DFlash max 6  68.3 tok/s

DSpark performs better across several less predictable or more mixed workloads:

C++ code
concept explanation
summarization
factual QA
creative writing
long code review

For example:

text

Concept explanation:
DSpark max 3  40.2 tok/s
DFlash max 6  30.4 tok/s

Factual QA:
DSpark max 3  44.5 tok/s
DFlash max 6  35.9 tok/s

Long code review:
DSpark max 3  37.8 tok/s
DFlash max 6  30.0 tok/s

So the updated result is not simply "DSpark is faster than DFlash."

A better interpretation is:

DSpark now has the better mixed-workload aggregate result on this hardware, while DFlash still has a substantial advantage on highly predictable structured output.

Acceptance rate still does not predict throughput

At their tested sweet spots:

text

DSpark max 3:
acceptance = 60.82%
effective  = 44.47 tok/s

DFlash max 6:
acceptance = 47.90%
effective  = 43.07 tok/s

DSpark now converts its higher acceptance rate into a small aggregate throughput advantage, but acceptance alone still does not explain performance.

Proposal cost, accepted-run depth, rejection position, target verification cost, runtime overhead and hardware scheduling all matter.

A strict DSpark-vs-DFlash architecture comparison would still require:

position-wise acceptance (P1, P2, P3, ...)
accepted-run-length distribution
per-round draft latency
target verification latency
identical training/checkpoint quality
identical vocabulary pruning and quantization

Without those controls, aggregate acceptance is insufficient to explain why one speculator is faster.

Draft-depth scaling

The updated results also show that deeper DSpark drafting still has diminishing returns:

text

max 2: 44.17 tok/s
max 3: 44.47 tok/s
max 4: 43.64 tok/s
max 5: 40.51 tok/s

Accepted-token count continues to increase:

text

max 2: 1,157
max 3: 1,268
max 4: 1,345
max 5: 1,373

but draft work grows much faster:

text

max 2: 1,610
max 3: 2,085
max 4: 2,469
max 5: 2,943

The clearest example is max 4 → max 5:

text

+474 proposed draft tokens
+28 accepted draft tokens

That additional draft work is not economical on this hardware, and aggregate throughput falls from 43.64 → 40.51 tok/s.

So although the updated DSpark path is substantially faster overall, deeper-draft efficiency remains an open optimization area.

Translation remains a failure mode

Translation is still poor for both external speculators:

text

No speculation: 28.6 tok/s
DSpark max 3:    19.4 tok/s
DFlash max 6:    21.4 tok/s
Native MTP max 2: 59.8 tok/s

DSpark acceptance on translation is only about 2.4% at max 3.

Translation has high token-level uncertainty: many different token sequences can express a valid translation, making exact speculative agreement difficult for an external draft model.

For translation on this setup, disable DSpark/DFlash speculation unless a better workload-specific configuration is found.

Suggested use

If your workload is mostly...	Recommended
Mixed general-purpose workload	DSpark max 3
C++ / QA / summaries / explanation / creative / code review	DSpark max 3
JSON / repeated patterns / constrained generation / code completion	DFlash max 6
Translation	Disable external speculative decoding
Native MTP available	Native MTP

DSpark's position is now more competitive than in the previous benchmark.

On this consumer dual-GPU setup it slightly wins the mixed aggregate, while DFlash remains the better specialist for highly predictable output.

Reproducibility notes

Results depend on:

target quantization
GPU placement
context length
KV-cache type
prompt/output distribution
sampling parameters
draft maximum
speculator quantization
llama.cpp revision
converter revision

For meaningful comparisons:

Keep the target GGUF fixed.
Use identical prompts and stopping conditions.
Compare total predicted tokens divided by total wall time.
Re-run DFlash as well as DSpark after runtime changes.
Run multiple repetitions before trusting small differences.

The current DSpark-vs-DFlash aggregate gap is only about 3.3%, so it should be treated as a small measured advantage rather than a universal performance claim.

Limitations

Draft model cannot generate output standalone.
Speculative decoding does not reduce target model VRAM/storage requirements.
Translation remains below the non-speculative baseline with DSpark and DFlash.
Deeper DSpark draft lengths still show rapidly diminishing returns on this hardware.
Native MTP remains substantially faster than either external speculator when available.
llama.cpp DSpark support is tied to an unmerged PR at the time of writing.
Vision/multimodal speculative decoding was not tested.
Measurements were performed on consumer NVIDIA GPUs and may differ significantly on RTX PRO / datacenter GPUs or other backends.
DSpark vs DFlash results also depend on checkpoint training quality and are not a controlled architecture-only comparison.

Credits

Target model: Google DeepMind — Gemma 4 31B-it
DSpark: DeepSeek AI — DeepSpec
Original speculator: RedHatAI — gemma-4-31B-it-speculator.dspark
Runtime integration: llama.cpp PR #26275
GGUF conversion and consumer-GPU benchmark: williamliao

License

This repository contains a converted speculator checkpoint. Use is subject to the licenses and terms of the original speculator checkpoint, the Gemma target model, and the relevant software projects. Review all upstream licenses before redistribution or commercial use.

williamliao/dspark_gemma4_31b-it-GGUF

Gemma 4 31B-it DSpark Speculator — GGUF

Status

Compatibility

Recommended settings

Benchmark

Change from the previous DSpark result

Per-workload (tok/s)

Interpretation

DSpark now slightly beats DFlash overall

DSpark and DFlash have different workload strengths

Acceptance rate still does not predict throughput

Draft-depth scaling

Translation remains a failure mode

Suggested use

Reproducibility notes

Limitations

Credits

License

williamliao/
dspark_gemma4_31b-it-GGUF