Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent sequential decoding limits inference efficiency.
We present Fast-dLLM v2 — a carefully designed block diffusion language model (dLLM) that efficiently adapts a pretrained AR model (Qwen2.5-1.5B-Instruct) into a diffusion-style decoder for parallel text generation.
Our approach introduces a novel decoding recipe incorporating a complementary attention mask and block diffusion mechanism, which together enable blockwise bidirectional context modeling while preserving the original AR training objectives and performance. To further enhance inference speed, we design a hierarchical caching mechanism: a block-level cache that stores historical context representations and a sub-block level cache that supports efficient parallel decoding within partially generated blocks.
✨ Key Innovations
Block Diffusion Mechanism + Complementary Attention Mask
Enables blockwise bidirectional context modeling without sacrificing AR objectives.
Hierarchical Caching
Block-level cache: Stores historical context representations across blocks.
Sub-block cache: Parallel decoding within partially generated blocks.
Token Shift Mechanism
Retains autoregressive characteristics while supporting bidirectional context within blocks.
Parallel Decoding Pipeline
Achieves up to 2.5× speedup over standard AR decoding without compromising quality.
🚀 Fast-dLLM v2 uses only ~1B tokens for fine-tuning — a 500× reduction vs. full-attention diffusion LLMs (Dream: 580B tokens) — while matching or surpassing AR baselines in accuracy.
You will need transformers, torch, and our custom generation function:
pip install transformers torch numpy
🚀 Quickstart
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model_name ="Efficient-Large-Model/Fast_dLLM_1.5B"45model = AutoModelForCausalLM.from_pretrained(6 model_name,7 torch_dtype="auto",8 device_map="auto",9 trust_remote_code=True10)1112tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)1314prompt ="Give me a short introduction to large language model."15messages =[16{"role":"system","content":"You are a helpful assistant."},17{"role":"user","content": prompt}18]1920text = tokenizer.apply_chat_template(21 messages,22 tokenize=False,23 add_generation_prompt=True24)25inputs = tokenizer([text], return_tensors="pt").to(model.device)2627# Fast-dLLM v2 parallel decoding28gen_ids = model.generate(29 inputs["input_ids"],30 tokenizer=tokenizer,31 max_new_tokens=512,32 small_block_size=8,33 threshold=0.9,34)3536response = tokenizer.decode(37 gen_ids[0][inputs["input_ids"].shape[1]:],38 skip_special_tokens=True39)40print(response)
📊 Performance & Benchmarks
▶ Real-time Throughput
Fast-dLLM v2 offers up to 2.54× higher throughput than Qwen2.5-7B-Instruct, without loss in quality.
Throughput Comparison
🏆 Benchmark Results
We compare Fast-dLLM v2 against AR baselines and previous diffusion LLMs on diverse tasks:
HumanEval, MBPP (code), GSM8K, Math (reasoning), IFEval (instruction), MMLU, GPQA (knowledge QA).
1B group: Fast-dLLM v2 (1.5B) achieves best average score: 45.0.
7B group: Fast-dLLM v2 (7B) achieves best average score: 60.3, surpassing LLaDA and Dream models.
Benchmark Results
📜 Citation
If you use Fast-dLLM v2 in your research or products, please cite:
bibtex
1@misc{wu2025fastdllmv2efficientblockdiffusion,
2 title={Fast-dLLM v2: Efficient Block-Diffusion LLM},
3 author={Chengyue Wu and Hao Zhang and Shuchen Xue and Shizhe Diao and Yonggan Fu and Zhijian Liu and Pavlo Molchanov and Ping Luo and Song Han and Enze Xie},
4 year={2025},
5 eprint={2509.26328},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2509.26328},
9}
📄 License
Released under Apache 2.0, following the base Qwen2.5 license.
