GGUF conversions of Koopah/Qwen3.6-35B-A3B-NVFP4-DSPARK-v2 for llama.cpp speculative decoding. See the parent card for the full model description, training status, acceptance evaluations and the vLLM/sglang serving numbers — this repo only carries the llama.cpp artifacts and deployment instructions.

Target + draft together are everything the deployment command below needs — the served pair is bit-for-bit what every benchmark on this card measured.

Recommended: the performance branch — KoopahTManiac/speed.llama.cpp (default branch). Upstream llama.cpp implements DSPARK (--spec-type draft-dspark) and serves this draft correctly; the performance branch is upstream plus a series of speed commits for this serving path — a fused in-graph truncation-sampler op, MoE finalize / weight-load fusions, batched elementwise launches, and host-path fixes — plus exact ratio acceptance (min(1, p/q) with exact residual sampling) for sampled drafting. Output quality is identical to running the target model alone, at any temperature (verified distributionally at N=3200 completions/config). The benchmark numbers below are from the performance branch.

--spec-draft-n-max is the drafted block length k (8 = fastest single-user and the trained block size — larger values are clamped to 8; 4–6 shift the throughput/latency trade at batch, same as the k-family curves on the parent card).

Optional — confidence-scheduled drafting (--spec-sched): the performance branch implements the DSpark paper's confidence scheduler using this draft's trained confidence head — hardware cost curve profiled at startup, sequential temperature scaling self-calibrated from live traffic and cached per model, causal expected-throughput draft admission. Zero configuration; add --spec-sched-adapt for adaptive depth + skip-drafting. On fast GPUs at short/mid context it measures as parity with fixed-depth drafting — use it where fixed-depth speculation loses to plain decoding (very long contexts, hard/creative content, weaker GPUs), and leave it off for batched serving (llama-server's host loop, not verify compute, is the batch ceiling).

Same protocol as the parent card's engine sections (16 concatenated GSM8K questions, output forced to 1024 tokens, temperature 0.7, 3-round average, RTX PRO 6000 Blackwell 96 GB), llama-server, BF16 drafter, dedicated single-user config (-np 1):

(Upstream measured under the identical protocol and config — it gains nothing from -np 1; its per-round overhead dominates regardless of slot count.)

llama.cpp wins plain single-stream decoding outright and reaches ~87% of vLLM with speculation; acceptance matches or beats vLLM on this workload (6.92 vs 6.84 mean accepted length), so the residual gap is per-round host/launch overhead, not draft quality.

Per-category verification on nvidia/SPEED-Bench (qualitative split, all samples, temp 0, OSL 1024, single user) via the in-tree speed-bench harness — baseline llama-server vs DSpark k=8 with Qwen3.6-35B-A3B-NVFP4-DSPARK-v2-BF16.gguf:

Known limitation — batch serving: llama-server's per-slot host loop (single-threaded sampling + stream writes) currently caps served batch throughput near 700 tok/s aggregate even though the raw engine scales to ~1,900 tok/s at bs64 (llama-batched-bench). Use vLLM or sglang for at-capacity serving; llama.cpp is the single-user/local option. At high slot counts also pass --cache-ram 0 (host prompt-cache eviction can livelock with this model's large per-slot state) and size -ub ≥ slots × 8 (the draft block is non-causal).