MiMo-V2.5-Pro DFlash Draft: BF16 GGUF for ik_llama.cpp
A GGUF conversion of the
DFlash block-diffusion draft for
MiMo-V2.5-Pro, built for speculative decoding in
ik_llama.cpp. This is the validation artifact for
ik_llama.cpp PR #2048.
Important: This is not a standalone model. It is a 5-layer draft that shares the target's token embedding and output head. It only produces meaningful output when launched as --model-draft alongside a full MiMo-V2.5-Pro target GGUF.
Files
| File | Size | Precision | SHA-256 |
|---|
mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf | 5,542,740,352 bytes | BF16 | e37bb1afdc5f8cc04df03b6f9d258be32afda44431191de22a056f4ab131a1e9 |
Matching target
Pair this draft with a MiMo-V2.5-Pro target GGUF. Validated against:
gghfez/MiMo-V2.5-Pro-ik_llama-unfused-GGUF, file MiMo-V2.5-Pro-IQ3_S-unfused.gguf (IQ3_S, about 349 GiB)
The validation used an unfused target. One fused-MoE conversion tested during the PR produced garbled target-only output, so it was excluded from the DFlash results. That observation does not establish that every fused MiMo conversion is incompatible.
Usage (ik_llama.cpp)
1llama-server \
2 -m MiMo-V2.5-Pro-IQ3_S-unfused.gguf \
3 --model-draft mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf \
4 -ngl 999 -ngld 999 \
5 -sm graph --max-gpu 4 \
6 -fa on -ctk q4_0 -ctv q4_0 -ctkd q4_0 -ctvd q4_0 \
7 --spec-type dflash:n_max=1,p_min=0.0,cross_ctx=16 \
8 -c 8192 --host 127.0.0.1 --port 8080
--spec-type dflash:... selects the DFlash speculator. n_max=1 was optimal in this test. cross_ctx=16 is a small ring buffer of recent target positions visible to the draft; larger values reduced acceptance sharply.-sm graph --max-gpu 4 was the best multi-GPU profile tested on 4×H200 NVL. Pinning the draft to one GPU with -devd added overhead under graph split.
Conversion metadata
| GGUF key | Value | Note |
|---|
dflash-draft.block_count | 5 | 5-layer draft |
dflash-draft.attention.head_count / head_count_kv | 128 / 8 | GQA |
dflash-draft.rope.dimension_count | 64 | head_dim 128 × partial_rotary_factor 0.5 (upper half is NoPE) |
dflash-draft.rope.freq_base | 10,000 | draft Q/K RoPE base from rope_theta |
dflash-draft.dflash.backbone_rotary_base | 5,000,000 | RoPE base used for captured target-context K |
dflash-draft.attention.value_scale | 0.612 | |
dflash-draft.dflash.block_size | 8 | diffusion block size |
dflash-draft.dflash.mask_token_id | 151669 | |
dflash-draft.dflash.target_layer_ids | [0, 15, 31, 47, 69] | captured target layers |
dflash-draft.dflash.n_target_features | 30,720 | target hidden_size 6144 × 5 layers |
dflash-draft.attention.sliding_window | 1024 | all-SWA pattern across the 5 layers |
| I/O contract | shared-target | token_embd and output reuse the target's tensors |
The file contains 63 tensors: five 12-tensor draft blocks plus dflash_fc, dflash_hidden_norm, and output_norm. It includes five blk.N.attn_sinks.weight tensors and does not contain token_embd or output tensors.
Validation
Validation was run remotely on 4×H200 NVL for ik_llama.cpp PR #2048. The draft was tested greedily at n_max=1, cross_ctx=16, with q4_0 KV cache on both target and draft.
| Prompt | Draft acceptance | DFlash decode (tok/s) | No-spec decode (tok/s) |
|---|
double-link-list fixture | 54.6% | 55.6 | 59.9 |
quick-sort fixture | 60.4% | 59.4 | 60.8 |
Conversion notes
- The source weights are the BF16 drafter in the
dflash/ subdirectory of XiaomiMiMo/MiMo-V2.5-Pro-FP4-DFlash.
- Target metadata and the tokenizer are read from the matching
MiMo-V2.5-Pro-FP4-DFlash repository root with --target-model-dir. The draft carries no embeddings of its own.
- Convert with
convert_hf_to_gguf.py from ik_llama.cpp PR #2048, using the DFlashDraftModel path and --outtype bf16.
1cd MiMo-V2.5-Pro-FP4-DFlash
2
3python3 ../ik_llama.cpp/convert_hf_to_gguf.py dflash \
4 --target-model-dir . \
5 --outtype bf16 \
6 --outfile mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf
Use ik_llama.cpp at PR #2048 or later. The final PR fix stores the draft's 10,000 RoPE base separately from the 5,000,000 backbone base. Older converter outputs must be reconverted rather than repaired by changing only rope.freq_base.
Troubleshooting low acceptance
- Confirm that the target produces coherent output by itself. A broken target conversion makes DFlash acceptance meaningless.
- Run an ik_llama.cpp build that includes PR #2048. The change includes runtime handling for attention sinks, value scaling, and MiMo2 target-feature capture in addition to GGUF metadata support.
- Start with
dflash:n_max=1,p_min=0.0,cross_ctx=16 and a greedy code-completion prompt comparable to the validation fixtures. The reported 55–60% rates are fixture-specific, not a universal acceptance baseline.
- Test the target with this published draft. If this file also gives very low acceptance, inspect the target conversion and runtime setup. If this file works, compare the converted draft against the metadata and 63-tensor contract above.
