This DFlash draft model is a joint retrain from Z-Lab and Modal, trained with 40k sequence length and sliding-window attention for improved long-context performance. It is mirrored across the following Hugging Face repositories:
This repository contains a DFlash draft model for Qwen/Qwen3.5-4B. It is not a standalone language model. It is intended to be paired with the target model in a speculative decoding server.
DFlash uses a lightweight block diffusion draft model to propose multiple tokens in parallel. The target model verifies those proposals, improving serving throughput while preserving the target model's output distribution.
DFlash Architecture
Quick Start
Installation
SGLang
Install a recent SGLang build with DFlash support:
uv pip install --upgrade "sglang[all]"
For best performance on Blackwell GPUs, use an SGLang build that includes DFlash, FA4/TRT-LLM attention, and FlashInfer support.
vLLM
For vLLM support, please refer to vllm-project/vllm#40898. We will update the PR to make it merge-ready soon.
Launch Server
This model should be used with an inference server that supports DFlash speculative decoding. An example SGLang deployment is:
Block size 8 is the recommended default for higher-concurrency serving. Block size 16 gives longer accept lengths and strong concurrency-1 throughput in most workloads.
Benchmark Results
We benchmarked DFlash against the autoregressive baseline and Qwen's built-in MTP draft path. DFlash reaches up to 4.60x speedup at concurrency 1 and 2.61x at concurrency 32. Across the benchmark suite, DFlash delivers higher throughput than MTP at every matched setting where both completed.
