openPangu-7B-Diffusion-DeepDiver is a 7B-parameter language model based on block diffusion large language models (Diffusion LLM), specifically trained and fine-tuned for multi-agent scenarios (including tool invocation, information retrieval, and multi-step decision-making). Its underlying architecture and inference pipeline follow the design of openPangu-R-7B-Diffusion (including block-wise denoising and bidirectional attention within blocks), thus maintaining consistent structures and interfaces for single-pass generation and parallel decoding.
For complete evaluations and training details, please refer to the technical report “DLLM Agent: See Farther, Run Faster” (arXiv:2602.07451v2).
openPangu-7B-Diffusion-DeepDiver: Agent model with a context length of 32k.
Key Features
Uses the same DLLM architecture and iterative inference pipeline as openPangu-R-7B-Diffusion.
Trained and fine-tuned with data and objectives tailored for in-depth agent research, making it more robust in multi-round tool invocation and planning tasks.
Introduces context-clean corruption and span-aware attention alignment to reduce noise propagation in multi-round agent dialogues and improve the reliability of tool invocation formats.
Inference
Context_Causal_Block_Diffusion_LLM
openPangu-7B-Diffusion-DeepDiver adopts context-causal block diffusion decoding, performing diffusion decoding block by block. Within each block, full attention is applied, while causal attention is used for preceding context. Once all tokens in a block are decoded, the entire block is stored in the historical KV cache with causal masking, and decoding proceeds to the first token of the next block.
Supports variable-length inference and KV caching.
Flexible context length, not limited by block size.
Supports both autoregressive and block diffusion decoding.
Uses confidence-threshold sampling, achieving up to 2.5× throughput improvement over standard autoregressive decoding.
Similar to Fast dLLMv2, small blocks can be configured within each block to balance throughput and performance, with optimal results typically at small block sizes of 4 or 8.
Inference in Agent Workflows
Integrated into the DeepDiver v2 Agent workflow, the model applies DLLM’s iterative denoising inference strategy for each round of tool generation.
Deepdiver v2 is a planner-centered MAS (Multi-Agent System) architecture that coordinates multiple executors.
The model is trained on 11k specially collected or synthesized agent trajectory datasets (including planner → seeker multi-agent interactions, real tool calls, and tool-return traces). These data aim to help the model learn to generate semantically consistent and format-compliant tool invocation instructions in multi-round interactions. See the “Agent-oriented Fine-tuning” section in the technical report for details.
Supervision Method
Cross-entropy losses for both diffusion and autoregressive models are jointly optimized during training, ensuring stability and preserving reliable left-to-right generation.
Masking and Attention Alignment
To address information contamination caused by diffusion when multi-round contexts and tool outputs are mixed, training applies context-clean corruption to mask irrelevant context segments and span-aware attention alignment within generation ranges. Experiments on agent datasets show that both techniques improve final information retrieval scores.
2. Model Architecture
openPangu-7B-Diffusion-DeepDiver
Architecture
Dense
Parameters (Non-Embedding)
7B
Number of Layers
34
Hidden Dimension
12800
Attention Mechanism
GQA
Number of Attention Heads
32 for Q, 8 for KV
Vocabulary Size
153k
Context Length
32k
Continued Training Tokens
700B
3. Evaluation Results
Table 1. Comparison results on a 110-question subset of BrowseComp-zh.
Method
Accuracy (%)
Tool Calls
Agent Rounds
Tool Failure Rate
AR Agent (autoregressive backbone)
15.5
7.5
14.8
1.9%
DLLM Agent (diffusion backbone)
15.5
6.7
13.0
6.4%
Although the final accuracy is comparable to AR on this subset, DLLM requires fewer tool calls and sub-agent rounds, and achieves about 30% average end-to-end latency reduction. However, DLLM shows a higher tool failure rate, indicating that it is still less stable than AR models.
4. Deployment and Usage
4.1 Environment Setup
Hardware Requirements
Atlas 800T A2 (64GB). For drivers and firmware, see:
[Atlas 800T A2].
The above configurations have been verified. Higher versions may be supported. Please submit an issue if you have questions.
4.2 Inference Examples
Below is a simple example of using openPangu-7B-Diffusion-DeepDiver with the transformers framework and the Deepdiver v2 Agent framework.
Loading and Running
Before running, modify generate.py to specify the model path.
bash
1cd inference
2python generate.py
For optimal throughput, set sampling parameters to alg="confidence_threshold", threshold=0.9, num_small_blocks=1, and choose an appropriate batch size based on hardware.
Download the Deepdiver v2 package (no model weights required) and install it following official documentation.
Copy env.template to config/.env, set MODEL_REQUEST_URL to the model service URL, and modify MODEL_NAME to match the deployed model name (default: local-diffusion-llm).
Start the MCP service:
python src/tools/mcp_server_standard.py
Send a query to Deepdiver v2:
python cli/demo.py -q "今天北京的天气怎么样？"
For more usage details, refer to the official repository.
Currently, openPangu-7B-Diffusion-DeepDiver has only been trained and tested within the Deepdiver v2 framework. It has not been adapted for other agent frameworks or tasks, and performance on other setups is not guaranteed.
5. License
When using the model or its outputs, please cite the technical report:
“DLLM Agent: See Farther, Run Faster” (arXiv:2602.07451v2).
Unless otherwise specified, openPangu-7B-Diffusion-DeepDiver is licensed under the OPENPANGU MODEL LICENSE AGREEMENT VERSION 1.0, which aims to promote the development of AI technologies. See the LICENSE file in the repository root for details.
6. Disclaimer
Due to inherent technical limitations and the nature of AI-generated content, Huawei makes no guarantees regarding the following:
The generated outputs may contain defects, inaccuracies, or inappropriate content, and do not represent Huawei’s views.
The model is not guaranteed to be 100% accurate, reliable, complete, timely, secure, error-free, uninterrupted, or stable.
The outputs do not constitute advice or decisions and do not guarantee authenticity, completeness, accuracy, legality, or usefulness. They cannot replace professional advice in medical, legal, or other domains. Users must make independent judgments, and Huawei assumes no responsibility.
7. Feedback
For suggestions or feedback, please submit an issue or contact: openPangu@huawei.com.
8. Citation
bibtex
1@article{zhen2026dllm,
2 title={DLLM Agent: See Farther, Run Faster},
3 author={Zhen, Huiling and Lin, Weizhe and Liu, Renxi and Han, Kai and Li, Yiming and Tian, Yuchuan and Chen, Hanting and Li, Xiaoguang and Li, Xiaosong and Chen, Chen and others},
4 journal={arXiv preprint arXiv:2602.07451},
5 year={2026}
6}
