Views
No views yet
--spec-type draft-mtp self-speculative
decoding for a real speedup, with no quality change to the base weights.| Mode | tok/s | Speedup | MTP acceptance | mean accepted len |
|---|---|---|---|---|
| AR (no MTP) | 66.6 | 1.00× | — | — |
| draft-mtp n_max=1 | 83.8 | 1.26× | 92.2% | 1.92 |
| draft-mtp n_max=2 | 82.8 | 1.24× | 82.5% | 2.65 |
| draft-mtp n_max=3 | 81.7 | 1.23× | 78.2% | 3.35 |
| draft-mtp n_max=4 | 75.9 | 1.14× | 68.1% | 3.72 |
--spec-draft-n-max 1, ~1.26×. (Acceptance is much higher on real text than on random
tokens — benchmark with a real prompt or you'll badly underestimate MTP.)1llama-server -m Ornith-1.0-35B-Q6_K-MTP.gguf -ngl 99 -c 32768 \
2 --spec-type draft-mtp --spec-draft-n-max 1 --port 8080draft-mtp speculative support.qwen35moe, 40 blocks) sibling that ships MTP
heads, following the cross-model graft approach published by
skinnyctax/Ornith-1.0-35B-Q6_K-Frankenstein-MTP-GGUF (MIT).
The 20 MTP head tensors (blk.40.*, incl. nextn.*) are appended to the base GGUF and metadata
patched (block_count +1, nextn_predict_layers=1).