You can deploy this model and start earning money today!

ToPo-ToPo/
DeepSeek-V4-Flash-MTP-bf16

Name: DeepSeek-V4-Flash-MTP-bf16
Availability: InStock

deepseek-ai/DeepSeek-V4-Flash

quantized

mit

8-bit

Views

No views yet

ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16

MTP drafter for speculative decoding with DeepSeek-V4-Flash on Apple Silicon (mlx-vlm), in bf16 (3.4 GB). This is a drafter, not a chat model — it is only useful as --draft-model.

Provenance (self-split from official weights)

Source: deepseek-ai/DeepSeek-V4-Flash (license: mit), whose mtp.* tensors live in a dedicated shard (model-00046-of-00046.safetensors)
Tool: mlx-vlm 0.6.8 — python -m mlx_vlm.speculative.drafters.deepseek_v4_mtp.split --model deepseek-ai/DeepSeek-V4-Flash --output .
Result: 45 tensors — decoder.* (35), hc_head (3), e_proj / h_proj (2 each), enorm, hnorm, norm

Usage

Pair it with a DeepSeek-V4-Flash body and run the MTP round loop (--draft-kind mtp):

bash

mlx_vlm.server --model ToPo-ToPo/DeepSeek-V4-Flash-0731-mlx-4bit \
               --draft-model ToPo-ToPo/DeepSeek-V4-Flash-MTP-bf16 --draft-kind mtp

Speculative decoding is lossless, so the body's outputs are unchanged.

This drafter also works with the newer -0731 body even though it is split from the earlier release — the two share hidden_size 4096. The MTP module bundled inside -0731 itself is a different architecture (three modules built around main_proj) that mlx-vlm 0.6.8 does not implement.