Qwen3.8-27B-MTPLX-Q8
MTPLX-compatible 8-bit conversion of Qwen/Qwen3.8-27B, built from the original BF16 checkpoint while preserving the model's native MTP weights in BF16.
Quantization
- Base model:
Qwen/Qwen3.8-27B
- Body: 8-bit affine
- Group size: 64
- MTP sidecar: native BF16
- Maximum verified MTP depth: 3
- Source format: native BF16 + MTP
Forge recipe:
1{
2 "body_bits": 8,
3 "body_dtype": "auto",
4 "body_group_size": 64,
5 "body_mode": "affine",
6 "mtp_policy": "keep_bf16"
7}
Verification Results
The model was verified locally with MTPLX Forge.
| Mode | Speed | vs AR | Acceptance |
|---|
| AR / depth 0 | 9.36 tok/s | 1.00× | — |
| MTP depth 1 | 18.84 tok/s | 2.01× | 100.00% |
| MTP depth 2 | 26.84 tok/s | 2.87× | 95.92% / 93.20% |
| MTP depth 3 | 29.03 tok/s | 3.10× | 97.38% / 90.09% / 83.67% |
Forge verdict:
All tested MTP depths passed the Forge quality verification gate.
Performance
The strongest verified configuration was MTP depth 3:
1AR: 9.36 tok/s
2MTP depth 3: 29.03 tok/s
3Speedup: 3.10×
The third speculative position was still accepted approximately 83.7% of the time.
These throughput numbers are hardware-specific and should primarily be used as relative measurements between AR and MTP on the same machine.
Q4 vs Q8
A separate 4-bit build of the same model was also tested on the same system.
| Quantization | AR | MTP D3 | MTP Speedup |
|---|
| Q4 | 15.74 tok/s | 44.34 tok/s | 2.82× |
| Q8 | 9.36 tok/s | 29.03 tok/s | 3.10× |
The Q8 build achieves a larger relative MTP multiplier, but the Q4 build has substantially higher absolute throughput because the smaller model body requires less memory bandwidth.
The Q8 variant is intended for users who prefer higher weight precision and can accept the additional memory usage and lower decode throughput.
MTP Acceptance
At the selected depth 3:
1Position 1: 97.38%
2Position 2: 90.09%
3Position 3: 83.67%
This high acceptance rate allows native MTP speculative decoding to provide a substantial speedup over standard autoregressive decoding.
Architecture
1Base architecture: Qwen3.8-27B
2MTPLX architecture: qwen3-next-mtp
3Body precision: 8-bit affine
4MTP precision: BF16
5Maximum MTP depth: 3
The native MTP weights were preserved from the original checkpoint and were not requantized.
Usage
This model is intended for MTPLX on Apple Silicon.
MTPLX can use the included mtplx_runtime.json metadata to determine the MTP contract, supported speculative depth, sampler configuration, and locally verified performance profile.
Related Model
For significantly higher throughput with a smaller memory footprint, see:
Kecven/Qwen3.8-27B-MTPLX-Q4
The Q4 build reached approximately 44.34 tok/s at MTP depth 3 on the same verification setup.
License
Apache-2.0, following the license of the original Qwen/Qwen3.8-27B model.
