---
pipeline_tag: text-generation
library_name: transformers
base_model: GSAI-ML/LLaDA-8B-Instruct
---

# $R^2$-dLLM-LLaDA

$R^2$-dLLM is a unified framework for reducing decoding redundancy in Diffusion Large Language Models (dLLMs) from both inference and training perspectives. This specific checkpoint is a redundancy-aware supervised fine-tuned version of [LLaDA-Instruct-8B](https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct).

- **Paper:** [$R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction](https://arxiv.org/abs/2604.18995)
- **Code:** [Official GitHub Repository](https://github.com/GATECH-EIC/R2-dLLM)

## Description

Diffusion Large Language Models (dLLMs) enable parallel token prediction but often suffer from high inference latency due to decoding redundancy. $R^2$-dLLM addresses this by:
1.  **Inference-time rules:** Aggregating local confidence and finalized predictions to avoid redundant decoding steps.
2.  **Redundancy-aware SFT:** Aligning the model with efficient decoding trajectories during training.

Experiments demonstrate that $R^2$-dLLM consistently reduces the number of decoding steps by up to 88% compared to existing decoding strategies, while maintaining competitive generation quality across different models and tasks.

## Citation

```bibtex
@article{du2026r,
  title={$R^{2}$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction},
  author={Du, Zhenbang and Xia, Kejing and Zhong, Xinrui and Fu, Yonggan and Oswald, Nicolai and Ji, Binfei and Khailany, Brucek and Molchanov, Pavlo and Lin, Yingyan},
  journal={arXiv preprint arXiv:2604.18995},
  year={2026}
}
```