The prepared MTP dataset used a maximum sequence length of 6,144 tokens. Its measured maximum was 4,830 tokens, with a mean of 2,969.98 and p95 of 3,939. The deterministic 90/10 split contained 3,658 training and 407 validation examples.

Setting	Value
Trainer	`speculators==0.6.0`
Verifier runtime	`vllm==0.22.1`
Target layer	32
Native MTP layers	1
Speculative steps trained	3
Epochs	3
Learning rate	1e-4
Step-weight beta	0.6
Seed	3407
Best checkpoint	epoch 3
Validation loss	0.0073809486
Step 0 / 1 / 2 loss	0.0031707955 / 0.0023736649 / 0.0018326155

Stitch isolation passed: all 426/426 non-MTP tensors remained value-identical to the verifier, while 13/15 MTP tensors changed during training.

Repository layout

text

.
├── config.json and model*.safetensors       # stitched HF model
├── gguf/
│   ├── embedded/                            # target + MTP in one GGUF
│   ├── target/                              # exact verifier GGUFs
│   └── sidecar/                             # trained MTP draft GGUFs
└── reports/                                 # isolation and acceptance evidence

The target GGUFs in this repository are not byte-identical to the normal GGUF release. Always pair a sidecar with the matching quantization from gguf/target/.

llama.cpp usage

The release was exported and accepted with llama.cpp commit 555881ebc8b0fc0402b30e09258a32a7bfd13c52, which supports Qwen3.5 draft-mtp.

Embedded Q4_K_M:

bash

llama-server \
  --model gguf/embedded/aurago-qwen35-4b-mtp-v1.Q4_K_M.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --ctx-size 8192

Separate Q4_K_M target and sidecar:

bash

llama-server \
  --model gguf/target/aurago-qwen35-4b-target.Q4_K_M.gguf \
  --spec-draft-model gguf/sidecar/aurago-qwen35-4b-mtp-v1-sidecar.Q4_K_M.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --ctx-size 8192

GPU offload and device flags are backend-specific. Confirm that both target and draft are placed on the intended device.

Verified results

NVIDIA A40

vLLM MTP-1, MTP-2, and MTP-3 each produced eight valid responses. The short MTP-1 and MTP-2 checks reported 100% draft acceptance. Embedded and sidecar Q8_0/Q4_K_M llama.cpp smoke tests all passed.

Intel Arc B580 with llama.cpp SYCL

Five measured runs followed one discarded warm-up at each context:

Mode	Context	Baseline tok/s	MTP tok/s	Gain	Draft acceptance	Faster runs
trained sidecar MTP-2	2K	59.88	98.37	+64.3%	100%	5/5
trained sidecar MTP-2	8K	51.59	90.68	+75.8%	100%	5/5
trained sidecar MTP-2	32K	24.72	68.94	+178.9%	100%	5/5

MTP-2 was chosen as the stable trained B580 setting. Results are specific to the tested llama.cpp SYCL build and hardware; Vulkan and other runtimes did not show the same benefit in every test.

404-case no-regression test

The baseline and trained MTP-2 sidecar each completed 404/404 cases with zero technical errors.

Metric	Baseline	trained MTP-2
Syntactically parseable	100%	100%
Schema-contract valid	95.61%	95.61%
Exact-case accuracy	97.77%	97.77%
Unauthorized safety actions	0%	0%
Parser/tool-error cases	0	0
Quality score	99.39%	99.39%

After removing nondeterministic tool_calls[].id values, all 404 predictions were semantically identical. The MTP no-regression gate passed.

Limitations

The speedups above are not universal. Benchmark each backend and device.
MTP increases memory use and may increase TTFT.
Sidecars require the exact matching target model and tokenizer.
Tool authorization and schema validation remain the responsibility of the host framework.
The 128K B580 case was removed from the local acceptance run because its TTFT was not operationally useful.

License

Apache License 2.0, following the upstream Qwen3.5 release.

antibyte/AuraGo-Qwen3.5-4B-MTP-v1

AuraGo-Qwen3.5-4B-MTP-v1

MTP training