Streaming-dLLM is a training-free acceleration framework for diffusion language models that supports efficient inference for models like Dream, LLaDA and LLaDA-1.5. We also provide a lightweight extension for block-causal diffusion models, with Open Pangu as an example on Ascend 910B.

Comparison of accuracy and throughput across different acceleration strategies. Our proposed method improves inference throughput while maintaining competitive accuracy compared to prior approaches.

Illustration of approximated suffix pruning. For each block, the nearest neighboring region following the current block is retained using a sliding window (red dashed box) and concatenated with the trailing position to form an approximate suffix region.

1. Clone this repo.

$ git clone https://github.com/xiaoshideta/Streaming-dLLM.git
$ cd Streaming-dLLM-main

1. Install all dependencies.

$ conda create -n stream-dllm python=3.10.19
$ pip install -r requirements.txt
$ conda activate stream-dllm

Your project structure should look like this:

|-- <Dream>
|-- <LLaDA-1.5>
|-- <OpenPangu>
|-- <Other>

Download the Dream model here.

Download the LLaDA model here.

Download the LLaDA-1.5 model here.

Download the Open Pangu model here.

cd Dream
bash eval_dream.sh

cd LLaDA-1.5
bash eval_llada.sh

Please first replace the llada-1.5 path with llada, and then execute the same script.

bash eval_llada.sh

Our method achieves 3.7×–13.3× speedup across all benchmarks over the vanilla backbone. Compared with the state-of-the-art acceleration method, it provides 1.5×–2.3× additional speedup on tasks with a generation length of 512. Meanwhile, our accuracy is comparable or slightly better, which demonstrates the effectiveness of our approach.

The Open Pangu adapter is placed under OpenPangu. Download the model weights first, then run:

cd OpenPangu/inference
python generate_streaming.py --model-path /path/to/openPangu-7B-Diffusion-Base --prompt "introduce the china"

For block-causal diffusion models, the spatial redundancy reduction in Streaming-dLLM degenerates into a block-topology-aware special case. We therefore apply the temporal decoding module to Open Pangu. Each cell reports accuracy and throughput in tokens per second (TPS).

If you find this work useful, please cite our paper:

@misc{xiao2026streamingdllmacceleratingdiffusionllms,
      title={Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding}, 
      author={Zhongyu Xiao and Zhiwei Hao and Jianyuan Guo and Yong Luo and Jia Liu and Jie Xu and Han Hu},
      year={2026},
      eprint={2601.17917},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2601.17917}, 
}

Part of our code is based on Fast-dLLM, LLaDA, Dream, and Open Pangu, thanks for their excellent work!