A DFlash speculative-decoding drafter for Qwen/Qwen3.6-35B-A3B and its derivatives — including Ornith-1.0-35B-AEON-Ultimate-Uncensored (a light RL post-train of that base). ~0.9 GB. Gives ~1.9× single-stream decode on a DGX Spark with no quality loss (lossless speculative decoding: the target model verifies every token).
What makes it different
This drafter is 8 layers of full attention — deliberately no sliding-window attention (SWA). That single design choice is the difference:
This drafter (AEON)
Typical SWA drafter (e.g. z-lab)
Layers
8 × full attention
6 × (5 sliding + 1 full), window 4096
KV page size
aligns with the target's pages
larger (windowed) → mismatch
Serves on the AEON vLLM image
yes, no patch
needs a kv_cache_utils patch
Why the no-patch part matters. Qwen3.6-35B-A3B / Ornith is a hybrid stack — full attention + GatedDeltaNet (Mamba) recurrent state. vLLM unifies KV-cache page sizes across layer types by scaling block_size, but Mamba/GDN state pages are shape-based and don't scale. A drafter with SWA layers introduces a larger KV page that can't be reconciled with the GDN pages → vLLM aborts at engine init with unify_kv_cache_spec_page_size … AssertionError (see issue #1). An all-full-attention drafter has no oversized page, so it loads cleanly on the stock AEON image — a true drop-in.
It also targets the right hidden states for this stack: DFlash reads target layers [1, 10, 19, 28, 37] of the 40-layer 35B-A3B model.
A note on KV cache (and why SWA doesn't help here)
This model serves with BF16 KV cache, and a sliding-window drafter does not change that. The BF16-KV requirement comes from two places, neither of which is the drafter's window type:
the vision tower (non-causal vision attention → FA2 backend only → no FP8 KV), and
DFlash's non-causal verification (block-parallel → FA2/FLEX only → no FP8 KV).
KV dtype is a property of the main model's attention backend, not the drafter's layers. SWA changes the drafter's own (tiny, windowed) KV footprint only — it can't unlock FP8 KV for the target.
Usage
Point any Qwen3.6-35B-A3B / Ornith deploy's --speculative-config at this drafter:
Validated on a DGX Spark (GB10) with the Ornith NVFP4 QuickStart. ⚠️ Keep --max-num-seqs ≤ 16 with DFlash on the Spark (unified-memory limit — see the QuickStart).
Performance
Head-to-head on Ornith-1.0-35B NVFP4 (DGX Spark GB10 · n=6 · max-num-seqs 16 · single stream):
Drafter
c=1 decode
Accept length
Draft accept rate
Patch needed
AEON (8L full-attn) — this repo
72.9 tok/s
3.71
45.1%
none
z-lab (6L SWA)
74.4 tok/s
3.35
39.2%
KV page-size hotfix
Speed is effectively tied (c=1 within ~2% — noise; this drafter is slightly ahead at c=8 aggregate). This drafter wins on acceptance — its 8 full-attention layers draft more accurately (3.71 vs 3.35 tokens accepted per step; 45% vs 39%) than the SWA variant's 6 layers, which trade accuracy for a cheaper forward pass. Combined with no patch required and no padded-Mamba-page overhead, it's the recommended default. Both are lossless (the target verifies every token); ~1.9× over plain decode either way.
