The MTP file is derived from the original Qwen3.8-27B MTP head and is compatible with Qwen3.8-27B derivatives. It is not trained on the Heretic weights.
llama.cpp
Requires a recent CUDA build with Qwen3.8, NVFP4, and draft-MTP support. Native FP4 execution requires NVIDIA Blackwell.
Tested on one GeForce RTX 5090 32 GB with llama.cpp b10453, CUDA 13.3 runtime, 262,144 allocated context, one slot, Flash Attention, K Q8_0 / V Q4_0 target KV, and Q4_0 draft KV.
Workload
Target tok/s
MTP accepted/generated
Acceptance
Mean draft length
Korean prose, 512 tokens
72.80
269/722
37.26%
2.12
Python code, 512 tokens
124.39
369/423
87.23%
3.62
Repetitive text, 512 tokens
135.76
381/388
98.20%
3.93
Load VRAM: approximately 27.22 GB.
Post-request VRAM: approximately 27.25 GB.
A 9,476-token retrieval prompt completed without OOM: 68.17 prompt tok/s and 21.09 generation tok/s.
Draft maxima 3, 5, and 7 were tested. 3 was retained because 5 and 7 reduced throughput on low-acceptance prose.
enable_thinking=false, Korean and English output, coding, authentication, start/stop, and VRAM release were tested through the OpenAI-compatible API.
These are single-system smoke measurements, not general benchmark claims. Throughput depends heavily on prompt structure, MTP acceptance, context length, runtime revision, and GPU state.
