DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference

Zhibin Wang, Wanchun Dou, Xue Li, Sheng Zhong, Chen Tian, Fuliang Liu, Zhonghui Zhang, Ketai Zhao, Yinxi Gao, Ziyan Zhou

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 15 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• fvliang/DART

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accuracy at the cost of multi-step autoregressive inference, resulting in high drafting latency and ultimately rendering the drafting stage itself a performance bottleneck. Inspired by diffusion-based large language models (dLLMs), we propose DART, which leverages parallel generation to reduce drafting latency. DART predicts logits for multiple future masked positions in parallel within a single forward pass based on hidden states of the target model, thereby eliminating autoregressive rollouts in the draft model while preserving a lightweight design. Based on these parallel logit predictions, we further introduce an efficient tree pruning algorithm that constructs high-quality draft token trees with N-gram-enforced semantic continuity. DART substantially reduces draftstage overhead while preserving high draft accuracy, leading to significantly improved end-toend decoding speed. Experimental results demonstrate that DART achieves a 2.03×-3.44× wallclock time speedup across multiple datasets, surpassing EAGLE3 by 30% on average and offering a practical speculative decoding framework. Code is released at https://github.com/fvliang/DART.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2601.19278")
get_code_for_paper("2601.19278")
have("2601.19278")

Connect an agent — have() is free.