GGUF conversion of the DSpark speculative-decoding draft model for google/gemma-4-26B-A4B-it.
This repository does not contain a standalone language model.
The GGUF file is a draft/speculator model and must be loaded together with a compatible Gemma 4 26B-A4B-it target model.
Update — llama.cpp PR #26275 / e107846
The GGUF files have been reconverted using llama.cpp PR #26275 commit e107846, which adds proper support for Speculators/SpecForge-format DSpark checkpoints.
This update is important because these checkpoints use a DFlash-style 1+N bonus-anchor block layout:
the first slot is a bonus/anchor token, not a prediction slot;
a block of size 7 therefore contains at most 6 actual draft predictions;
llama.cpp now records this using dflash.bonus_anchor;
DSpark prediction/confidence is read from the mask positions rather than the anchor;
reduced draft vocabularies can be expanded correctly by the converter.
Because the GGUF representation itself changed, older conversions should be replaced by GGUF files produced with the updated converter.
Practical result
On the same RTX 5070 Ti + RTX 5060 Ti consumer-GPU system used for the earlier tests, the updated conversion improved DSpark throughput substantially without materially changing draft acceptance.
The main remaining observation is unchanged:
DFlash is still faster overall on this fast MoE target, but the gap is smaller than with the previous DSpark conversion.
Status
DSpark support is being developed in:
ggml-org/llama.cpp#26275
The results below use the PR branch including:
e107846f1963bd24ef7940c20a4ae4f8d03070b9
Until the PR is merged, use a llama.cpp build containing this commit or a newer revision of the PR.
Compatibility
Component
Requirement
Target architecture
Gemma 4 26B-A4B
Recommended target
google/gemma-4-26B-A4B-it or compatible GGUF
Draft format
DSpark / SpecForge
Runtime
llama.cpp PR #26275 with e107846 or later
Standalone use
No
Trained block size
7 (1 anchor + 6 draft slots)
Effective maximum draft length
6
Recommended draft maximum
5
The target model and speculator must use compatible tokenizer and vocabulary metadata.
Recommended settings
text
1speculative type: DSpark
2draft max: 5
max 5 remains the best practical configuration in this benchmark.
The updated runtime correctly clamps this checkpoint to an effective maximum of 6 draft tokens. Values larger than 6 therefore do not create additional draft positions.
For example, max 6, max 7, and max 15 all produced exactly:
The wall-time differences between those runs should therefore be treated as normal run-to-run variance rather than a real benefit from setting a larger n_max.
max 6, 7, and 15 execute the same effective draft workload because the checkpoint is capped at six prediction slots. Their wall-time differences are measurement variance and should not be interpreted as draft-depth scaling.
That is a large increase in draft work for only a very small increase in accepted tokens.
For this hardware and target, the sixth draft position does not pay for its execution cost across the mixed workload.
Before vs. after e107846
The old GGUF conversion used the earlier DSpark handling before the SpecForge bonus-anchor layout was represented explicitly.
Configuration
Acceptance
Wall time
Effective rate
Old conversion — max 5
43.56%
18.70 s
108.8 tok/s
New e107846 conversion — max 5
43.56%
16.75 s
121.5 tok/s
This is approximately:
text
1108.8 → 121.5 tok/s
2≈ +11.7%
while aggregate acceptance remains exactly:
43.56%
This is an interesting result because it suggests that the improvement is primarily related to the corrected conversion/runtime handling rather than improved prediction quality.
The new implementation also prevents configurations such as max 7 from performing an extra invalid/unused prediction position.
Previously:
text
1max 7:
234.09% acceptance
327.31 s
491.4 tok/s
With e107846, values above the trained six prediction positions are correctly capped.
Per-workload — updated DSpark max 5
Workload
Acceptance
tok/s
Python code
73.2%
198.7
C++ code
61.4%
176.7
Concept explanation
28.9%
107.2
Summarization
30.0%
115.5
Factual QA
28.9%
108.6
Translation
9.4%
58.8
Creative writing
9.6%
54.4
Stepwise math
50.6%
157.5
JSON output
79.2%
205.7
Long reasoning
34.1%
110.6
Repeated pattern
86.6%
231.8
Code completion
83.7%
223.2
Long code review
20.4%
85.0
The workload dependence remains very strong.
Highly predictable outputs such as JSON, repeated patterns, and code completion benefit substantially from speculative decoding.
Translation, creative writing, and long code review continue to have relatively low acceptance and can perform worse than normal target decoding.
DSpark vs. previous DFlash result
The previous benchmark on this system measured:
Mode
Effective rate
No speculation
98.9 tok/s
Old DSpark max 5
108.8 tok/s
New DSpark max 5 (e107846)
121.5 tok/s
DFlash max 6
145.7 tok/s
Using those measurements:
text
1Old DSpark vs baseline: ~1.10x
2New DSpark vs baseline: ~1.23x
3DFlash vs baseline: ~1.47x
The DFlash advantage over DSpark therefore decreases from roughly:
145.7 / 108.8 = 1.34x
to approximately:
145.7 / 121.5 = 1.20x
So the corrected DSpark conversion/runtime path meaningfully narrows the gap, but DFlash still leads overall on this hardware.
Important comparison caveat
The baseline and DFlash figures above come from the earlier benchmark revision, while the updated DSpark result uses PR #26275 commit e107846.
They are useful for understanding the approximate trend, but a strict performance comparison should rerun baseline, DSpark, and DFlash on the exact same llama.cpp revision.
Interpretation
Gemma 4 26B-A4B is a fast MoE target.
Only a subset of parameters is active for each token, so target decoding is already relatively inexpensive. This changes the economics of speculative decoding: draft-model overhead becomes a larger fraction of total generation time.
The updated results suggest several things.
1. The SpecForge layout fix matters
The new conversion/runtime path improves max-5 throughput from:
108.8 → 121.5 tok/s
without changing aggregate acceptance.
So at least part of the previous performance loss was related to runtime/conversion handling rather than draft-model prediction quality.
2. Longer drafts are still not automatically better
Max 6 generates about 15% more draft tokens than max 5:
3166 → 3639
but gains only:
1379 → 1403 accepted tokens
For this consumer-GPU setup, that trade is not worthwhile.
3. DSpark performance remains workload-dependent
DSpark performs very well on:
JSON
repeated patterns
code completion
Python/C++ generation
but poorly on:
translation
creative writing
long code review
This is consistent with speculative decoding being most effective when upcoming tokens are predictable.
4. DFlash remains faster overall
Even after the SpecForge correction, the previous DFlash result remains substantially faster than the updated DSpark result.
However, the gap is now closer to ~20% rather than ~34%.
This makes the remaining difference especially interesting: it may reflect DSpark runtime overhead, hardware characteristics, checkpoint behavior, or other implementation details rather than simply an incorrect GGUF conversion.
Further apples-to-apples testing on the same llama.cpp revision is needed before assigning the remaining gap to any single cause.
Suggested use
Situation
Recommendation
General mixed workload
DSpark max 5
DFlash available for this target
DFlash is currently faster overall
JSON / repeated patterns / code completion
DSpark performs very well
Python generation
DSpark is highly competitive
Translation / creative prose
Consider disabling speculative decoding
Testing DSpark runtime behavior
Use max 5 and record per-task acceptance
n_max > 6
No benefit; runtime clamps to trained maximum
Reproducibility notes
Speculative-decoding performance depends heavily on:
target quantization;
draft quantization;
GPU assignment;
PCIe topology;
context length;
KV-cache type;
batch and ubatch size;
sampling parameters;
draft confidence threshold;
draft maximum;
prompt/output distribution;
llama.cpp revision.
For meaningful comparisons:
use the same target GGUF;
use identical prompts and stopping conditions;
use the same llama.cpp commit;
compare total predicted tokens divided by total wall time;
record both acceptance and total draft work;
repeat runs before trusting small differences.
Aggregate acceptance alone is not enough to predict throughput.
A configuration can have good acceptance while still losing performance if the extra draft computation costs more than the accepted tokens save.
Limitations
This GGUF is a speculator only and cannot generate standalone.
It does not reduce the VRAM/storage requirement of the target model.
Results were measured on consumer NVIDIA GPUs.
Performance may differ substantially on RTX PRO, RTX 6000, A100, H100, B200, or other accelerators.
DFlash results should be rerun on the same llama.cpp commit for a strict post-e107846 comparison.
Translation and creative writing remain poor speculative-decoding workloads in this test.
Vision/multimodal speculative decoding was not tested.
PR #26275 is still under development at the time of writing.
Credits
Target model: Google DeepMind — Gemma 4 26B-A4B-it
Original DSpark checkpoint: makora-ai/gemma4-26b-a4b-dspark
Runtime integration: ggml-org/llama.cpp PR #26275
SpecForge support / bonus-anchor handling: llama.cpp commit e107846
GGUF conversion and consumer-GPU benchmarks: williamliao
License
This repository contains a converted speculator checkpoint.
Use is subject to the licenses and terms of the original speculator checkpoint, the Gemma target model, and the relevant software projects. Review all upstream licenses before redistribution or commercial use.
