Views
No views yet
0642d218) with one metadata fix and bit-identical weights: config.json's
rope fields are corrected to full-width RoPE (partial_rotary_factor: 1.0,
rope_type: default, mrope fields removed).partial_rotary_factor: 0.25 and mrope fields from the target
model's text config, but the DeepSpec training code
never reads them — its Qwen3 attention applies RoPE across the full head dim
(Qwen3RotaryEmbedding + rotate_half from dense modeling_qwen3).{"method": "dspark"} — honor the field, rotate only 64 of 256 head dims, and produce
position-degenerate drafts:| Serving setup (target: unsloth/Qwen3.6-27B-NVFP4, block 15) | Draft acceptance |
|---|---|
| vLLM ≥ 0.25 + original config | ~1.4% (net slowdown) |
| vLLM ≥ 0.25 + this config | ~12.6% (≈2.9 tokens/step) |
| (sibling 0.8B head, dense bf16 target, same fix) | 6–8% → 25.1% |
--spec-type draft-dspark) is unaffected — its GGUF conversion already uses
full-width rope. For GGUF files, use the original repo.1vllm serve unsloth/Qwen3.6-27B-NVFP4 \
2 --max-num-seqs 8 --max-num-batched-tokens 8448 \
3 --enable-prefix-caching --mamba-block-size 256 \
4 --speculative-config '{"method":"dspark","model":"stepnivlk/Qwen3.6-27B-DSpark","num_speculative_tokens":15}'num_speculative_tokens: 15 is the trained block size (see the original model card —
lower values discard most of the speedup). Raise --max-num-batched-tokens if vLLM reports a
negative max_num_scheduled_tokens (15 draft slots per sequence add up).satgeze/Qwen3.6-27B-DSpark @ 0642d2182fb557ac4efee5c77b22ecb5447137ea
(model.safetensors, sha-identical). All training credit to the original author; head trained
with DeepSpec against Qwen3.6-27B, warm-started from the z-lab DFlash head.