MiMo2 MTP support is now available in upstream llama.cpp through PR #26228. No custom fork or separate draft model is required. Use a llama.cpp build containing merge commit 64d528be or later.

Files

MiMo-V2.5-IQ3_S.gguf
- IQ3_S-style target quant with appended BF16 MTP tensors.
- Made to replicate AesSedai's MiMo-V2.5 IQ3_S quantization recipe, using AesSedai's imatrix.
- Recipe summary: dense/infra Q6_K, expert gate/up/down IQ2_S / IQ2_S / IQ3_S, MTP tensors appended as BF16.
IQ2_XXS/
- 17-shard IQ2_XXS / Q8-infra + BF16 MTP GGUF set.
- Recipe summary: dense/infra Q8_0, expert gate/up/down IQ2_XXS, nextn.eh_proj BF16, norms/biases left F32.
Q4_K/
- 17-shard Q4_K / Q8-infra + BF16 MTP GGUF set.
- Recipe summary: dense/infra Q8_0, expert gate/up/down Q4_K, nextn.eh_proj BF16, norms/biases left F32.
Q6_K/
- 17-shard Q6_K / Q8-infra + BF16 MTP GGUF set.
- Recipe summary: dense/infra Q8_0, expert gate/up/down Q6_K, nextn.eh_proj BF16, norms/biases left F32.

Usage

Build current upstream llama.cpp:

bash

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-server -j

For CUDA, add -DGGML_CUDA=ON to the CMake configuration. For Metal, the default macOS build enables Metal automatically.

Run MTP from the same GGUF:

bash

./build/bin/llama-server \
  --model /path/to/MiMo-V2.5-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 1 \
  --temp 0.6 \
  -np 1 \
  -ngl 99

For split GGUFs, pass the first shard to --model.

Notes

Recommended starting point: --spec-draft-n-max 1.
Use --spec-type draft-mtp; older notes using --spec-type mtp refer to the previous experimental implementation.
This is a same-GGUF draft path. Do not pass a separate draft model.
The GGUF must include the MiMo MTP / nextn tensors for MTP drafting. Trunk-only MiMo GGUFs continue to work for target-only inference.
-np 1 is the recommended validation path.
Image/audio input from the original omnimodal MiMo-V2.5 release is not available through this GGUF text path.

Tested Setups

Apple Silicon:

Apple M3 Max, Metal, 128 GB unified memory.
MiMo-V2.5 IQ3_S GGUF with appended BF16 MTP tensors.
Smoke-tested with --spec-type draft-mtp --spec-draft-n-max 1.

CUDA:

RTX 3090 + Tesla P40.
DDR5 system memory with CPU/GPU hybrid offload.
MiMo-V2.5 IQ2_XXS / Q8-infra GGUF with BF16 MTP tensors.
Tested with --spec-type draft-mtp --spec-draft-n-max 1.

Performance

MTP speedup is workload-dependent. These measurements were collected with the IQ2_XXS / Q8-infra + BF16-MTP GGUF on an RTX 3090 + Tesla P40 system using the CUDA backend and --spec-draft-n-max 1:

workload	target only	MTP	change	acceptance
repeated warm prompt	about `22.2 t/s`	`24.2-24.4 t/s`	about `+9-10%`	`84.1%`
cached chat at about 17.5k context	`21.20 t/s`	`23.65 t/s`	`+11.6%`	`84.1%`

A deterministic 256-token high-margin test produced an identical committed token sequence between target-only and MTP generation, with 127/127 draft tokens accepted.

On Apple M3 Max / Metal with the IQ3_S GGUF, earlier smoke tests showed the same general behavior: high-agreement prompts gained around 10-20%, while lower-agreement prompts were closer to baseline.

Previous Fork

The earlier tnhnyzc/llama-mimo-mtp fork is retained for development history and reference. New deployments should use upstream llama.cpp.

tnhnyzc/MiMO-V2.5-MTP-GGUF

MiMo-V2.5 MTP GGUF