This is a draft model only. It cannot be used by itself. Pair it with a compatible Gemma 4 12B target model and run it with DSpark speculative decoding in llama.cpp.

[!IMPORTANT] Experimental, and not directly comparable to other DSpark cards in this collection. This model uses llama.cpp PR #25549 — the native Gemma 4 DSpark implementation — not PR #26275 (speculators-format checkpoint support) used for the 26B-A4B and 31B DSpark cards. Different PR, different upstream checkpoint, different block size, and this benchmark also enables confidence gating (--spec-draft-p-min 0.60) that the other cards do not use. Don't read the acceptance/speedup numbers below as an apples-to-apples architecture comparison against those cards — the delta could easily come from any of those four variables, not from target size.

Files

File	Purpose
`gemma4_12b-DSpark-Q4_K_M.gguf`	Quantized DSpark draft model
`gemma4_12b-DSpark-BF16.gguf`	BF16 source conversion, when provided

Recommended target: google/gemma-4-12B-it (may be quantized). Draft and target must use the same tokenizer/model family.

Conversion

bash

hf download google/gemma-4-12B-it --local-dir Gemma4-12B-it
hf download deepseek-ai/dspark_gemma4_12b_block7 --local-dir dspark_gemma4_12b_block7

python convert_hf_to_gguf.py dspark_gemma4_12b_block7 \
  --outtype bf16 \
  --target-model-dir Gemma4-12B-it \
  --outfile gemma4_12b-DSpark-BF16.gguf

llama-quantize gemma4_12b-DSpark-BF16.gguf gemma4_12b-DSpark-Q4_K_M.gguf Q4_K_M

--target-model-dir must point at the target checkpoint (Gemma4-12B-it), not the DSpark directory — the draft checkpoint doesn't ship its own tokenizer.

llama.cpp usage

bat

llama-server.exe ^
  -m "gemma-4-12B-it-Q4_K_M.gguf" ^
  --model-draft "gemma4_12b-DSpark-Q4_K_M.gguf" ^
  --host 0.0.0.0 ^
  --port 18010 ^
  --device-draft CUDA1 ^
  --threads 12 ^
  --threads-batch 12 ^
  --split-mode layer ^
  --tensor-split 14,12 ^
  --ctx-size 65536 ^
  --no-mmap ^
  -ngl -1 ^
  --flash-attn on ^
  --ubatch-size 256 ^
  --batch-size 2048 ^
  --cache-type-k q8_0 ^
  --cache-type-v q8_0 ^
  --temp 1.0 ^
  --top-k 64 ^
  --top-p 0.95 ^
  --reasoning off ^
  --fit off ^
  --parallel 1 ^
  --spec-type draft-dspark ^
  --spec-draft-n-max 7 ^
  --spec-draft-p-min 0.60

--spec-draft-n-max 7 matches the checkpoint's block size 7. --spec-draft-p-min 0.60 gates low-confidence proposals before verification — see Interpretation for why this matters for the acceptance numbers below.

Benchmark

Setup: RTX 5070 Ti 16GB + RTX 5060 Ti 16GB, target Gemma 4 12B-it Q4_K_M, context 65,536, temp 1.0/top-k 64/top-p 0.95, reasoning off, 1 parallel slot, mtp-bench.py, mostly 192 tokens/test. Draft only tested at Q4_K_M — no BF16/Q8_0 comparison was run (see Limitations).

Not directly comparable to the BF16 target benchmarks in the upstream PR, or to the 26B-A4B / 31B DSpark cards (different PR/checkpoint/config — see the warning above).

Test	No draft	DSpark	Speedup	Acceptance
code_python	74.9	124.8	1.67×	81.0%
code_cpp	75.0	112.1	1.49×	77.5%
explain_concept	74.4	89.2	1.20×	72.3%
summarize	75.5	99.0	1.31×	58.6%
qa_factual	74.9	91.1	1.22×	58.6%
translation	75.5	81.8	1.08×	53.6%
creative_short	75.7	75.4	1.00×	73.9%
stepwise_math	74.8	155.4	2.08×	83.1%
json_output	74.4	184.5	2.48×	94.7%
long_reasoning	74.5	102.2	1.37×	81.5%
repeat_pattern	74.7	196.8	2.63×	93.1%
code_completion	74.2	140.9	1.90×	83.2%
long_code_review	72.5	83.2	1.15×	75.4%

Aggregate

Metric	No draft	DSpark
Predicted tokens	2,015	2,016
Draft tokens	0	1,787
Accepted draft tokens	0	1,410
Aggregate acceptance	—	78.9%
Wall time	52.11 s	18.80 s
Effective throughput	38.67 tok/s	107.23 tok/s
Speedup	1.00×	2.77×

Baseline was stable at 72.5–75.7 tok/s across all tasks regardless of workload — the variation in the DSpark column is entirely from drafting behavior, not target-side variance.

Interpretation

The strongest gains were on predictable, structured output: repeat pattern (2.63×), JSON (2.48×), stepwise math (2.08×), code completion (1.90×), Python (1.67×). Open-ended generation benefited less — creative_short was essentially unchanged, translation gained only ~8%. This matches the pattern seen on the other Gemma 4 DSpark cards: structured/constrained token sequences are easier for a small draft model to predict than open-ended or high-entropy text.

On the 78.9% aggregate acceptance specifically: this is much higher than the 26B-A4B card's 43.56% and the 31B card's 60.82%, but this run also used --spec-draft-p-min 0.60, which the other two didn't. Confidence gating filters out low-confidence draft proposals before they're sent for verification, which mechanically inflates the acceptance ratio (you're only counting the proposals the draft was already confident about). Some of this run's advantage may be genuine — different PR, different checkpoint — but some of it is very plausibly just p_min doing its job. This isn't validated either way without an ablation (same checkpoint, p_min off vs. 0.60), so treat the 78.9% number as "this run's config," not "this checkpoint's ceiling acceptance."

High acceptance still doesn't fully explain the 2.77× end-to-end speedup — draft execution cost, verification batching, quantization, and cross-GPU transfer all factor in too.

Notes

DSpark performance depends heavily on workload predictability.
High acceptance does not guarantee proportional throughput.
The upstream PR recommends keeping the draft in BF16; this benchmark only tested Q4_K_M, so no local evidence either supports or contradicts that — if quantization sensitivity matters to your deployment, test BF16/Q8_0/ Q4_K_M yourself before assuming Q4_K_M is sufficient.
Gemma 4 DSpark support remains experimental and may change while PR #25549 is under review.

Limitations

Draft model cannot generate output standalone.
Only Q4_K_M draft quantization was benchmarked — no BF16/Q8_0 comparison.
Results are single-configuration, single-hardware local measurements, not a controlled comparison against the 26B-A4B/31B DSpark cards (different PR, checkpoint, and --spec-draft-p-min setting — see the notice above).
Confidence-gating contribution to the acceptance rate has not been isolated via ablation.
Vision/multimodal behavior not tested.

Credits

Original target: google/gemma-4-12B-it
Original DSpark checkpoint: deepseek-ai/dspark_gemma4_12b_block7
llama.cpp Gemma 4 DSpark implementation: PR #25549

This repository only provides a GGUF conversion/quantization. Please follow the licenses and usage terms of the original target model, draft model, and llama.cpp.

williamliao/dspark_gemma4_12b-GGUF

Gemma 4 12B DSpark GGUF