D2F LoRA adapter for LLaDA-8B-Instruct
This repository contains the LoRA adapter for the GSAI-ML/LLaDA-8B-Instruct model, trained using the Discrete Diffusion Forcing (D2F) method.
This adapter allows the LLaDA-8B-Instruct diffusion LLM (dLLM) to achieve inference speeds that are significantly faster than both its original version and leading autoregressive (AR) models like LLaMA3, while maintaining comparable output quality.
Method: Discrete Diffusion Forcing (D2F)
Diffusion LLMs (dLLMs) have long promised ultra-fast parallel decoding, but this potential was historically crippled by two main bottlenecks:
- KV Cache Incompatibility: Their bidirectional attention mechanism prevented the use of the Key-Value Cache, a critical optimization in AR models.
- Strict Inter-Block Dependency: Previous attempts at block-based generation required each block to be fully generated before starting the next, preventing true parallelism.
D2F solves these issues with a novel hybrid approach:
-
Hybrid Architecture: D2F reframes text generation as a block-autoregressive process.
- Within a block: Attention remains bidirectional to capture rich local context.
- Between blocks: Attention is made causal, allowing the model to be fully compatible with the standard KV Cache.
-
Pipelined Parallel Decoding: D2F uses an efficient training and inference strategy.
- Training: It uses Asymmetric Distillation, where a D2F student model learns to mimic a powerful bidirectional teacher model, efficiently transferring its capabilities to the fast, cache-friendly architecture.
- Inference: It enables a dynamic pipelined parallel decoder. New text blocks are added to the pipeline as soon as their predecessors are only partially complete. This creates an asynchronous workflow that maximizes GPU utilization and dramatically boosts throughput.
How to Use
⚠️ Important: This is a LoRA adapter and requires the official D2F codebase for inference.
For detailed instructions and code, please refer to the official GitHub repository:
