You can deploy this model and start earning money today!

fvliang/
qwen1.7b-dart

Name: qwen1.7b-dart
Availability: InStock

safetensors

llama

apache-2.0

Views

No views yet

DART

DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference

[Paper will come soon]

Overview

DART is a new speculative decoding approach for Large Language Models (LLMs) inference, which is inspired by diffusion-based Large Language Models (dLLMs). DART surpasses EAGLE3 by 30% on average, achieving up to 65% improvement on certain code-centric workloads.

DART's drafting requires only a single forward pass of a single transformer layer and a fast cpp-based tree search to build the draft token tree, resulting in extremely low drafting cost while preserving relatively high $\tau$ (Average Acceptance Length).

Key Features

Fast drafting Forward: Produces multiple logits simultaneously with 1 forward of 1 layer.
Fast Tree Search: Uses n-gram–based tree search to build the final draft tree (C++ based).
Low Drafting Cost: Results in extremely low drafting cost for efficient inference.
relatively high $\tau$: Average Acceptance Length is competitive with EAGLE3.

Quick Start

Installation

bash

git clone https://github.com/fvliang/DART.git
cd DART
curl -LsSf https://astral.sh/uv/install.sh | sh (optimal, if you don't have a uv)
uv sync
uv pip install -e .

Model Weights (HuggingFace)

Base Model	DART Weights	N-gram Model
Qwen3-1.7B	fvliang/qwen1.7b-dart	fvliang/dart-qwen3-ngram
Qwen3-4B	fvliang/qwen4b-dart	fvliang/dart-qwen3-ngram
Qwen3-8B	fvliang/qwen8b-dart	fvliang/dart-qwen3-ngram
Qwen3-14B	fvliang/qwen14b-dart	fvliang/dart-qwen3-ngram
Qwen3-32B	fvliang/qwen32b-dart	fvliang/dart-qwen3-ngram

Model Weights (ModelScope)

Base Model	DART Weights	N-gram Model
Qwen3-1.7B	fvliang/Qwen3-1.7B-dart	fvliang/dart-qwen3-ngram
Qwen3-4B	fvliang/Qwen3-4B-dart	fvliang/dart-qwen3-ngram
Qwen3-8B	fvliang/Qwen3-8B-dart	fvliang/dart-qwen3-ngram
Qwen3-14B	fvliang/Qwen3-14B-dart	fvliang/dart-qwen3-ngram
Qwen3-32B	fvliang/Qwen3-32B-dart	fvliang/dart-qwen3-ngram

fvliang/qwen1.7b-dart