Visual Autoregressive Modeling for Instruction-Guided Image Editing
Recent advances in diffusion models have brought remarkable visual fidelity
to instruction-guided image editing. However, their global denoising process
inherently entangles the edited region with the entire image context, leading
to unintended spurious modifications and compromised adherence to editing
instructions. In contrast, autoregressive models offer a distinct paradigm by
formulating image synthesis as a sequential process over discrete visual
tokens. Their causal and compositional mechanism naturally circumvents the
adherence challenges of diffusion-based methods. In this paper, we present
VAREdit, a visual autoregressive (VAR) framework that reframes image editing as
a next-scale prediction problem. Conditioned on source image features and text
instructions, VAREdit generates multi-scale target features to achieve precise
edits. A core challenge in this paradigm is how to effectively condition the
source image tokens. We observe that finest-scale source features cannot
effectively guide the prediction of coarser target features. To bridge this
gap, we introduce a Scale-Aligned Reference (SAR) module, which injects
scale-matched conditioning information into the first self-attention layer.
VAREdit demonstrates significant advancements in both editing adherence and
efficiency. On standard benchmarks, it outperforms leading diffusion-based
methods by 30\%+ higher GPT-Balance score. Moreover, it completes a
512times512 editing in 1.2 seconds, making it 2.2times faster than the
similarly sized UltraEdit. The models are available at
https://github.com/HiDream-ai/VAREdit.
68a7ceb139413c456c05b0bf
Qingyang Mao
68a7ceb139413c456c05b0c0
Qi Cai
claimed_verified
2025-08-22T07:21:57.195Z
63743880e75ef9e965353987
/avatars/a2b6401d7515c9c171ceab2ce7a8c406.svg
Qi Cai
user
cai-qi
68a7ceb139413c456c05b0c1
Yehao Li
68a7ceb139413c456c05b0c2
Yingwei Pan
68a7ceb139413c456c05b0c3
Mingyue Cheng
68a7ceb139413c456c05b0c4
Ting Yao
68a7ceb139413c456c05b0c5
Qi Liu
68a7ceb139413c456c05b0c6
Tao Mei
Visual Autoregressive Modeling for Instruction-Guided Image Editing
Recent advances in diffusion models have brought remarkable visual fidelity
to instruction-guided image editing. However, their global denoising process
inherently entangles the edited region with the entire image context, leading
to unintended spurious modifications and compromised adherence to editing
instructions. In contrast, autoregressive models offer a distinct paradigm by
formulating image synthesis as a sequential process over discrete visual
tokens. Their causal and compositional mechanism naturally circumvents the
adherence challenges of diffusion-based methods. In this paper, we present
VAREdit, a visual autoregressive (VAR) framework that reframes image editing as
a next-scale prediction problem. Conditioned on source image features and text
instructions, VAREdit generates multi-scale target features to achieve precise
edits. A core challenge in this paradigm is how to effectively condition the
source image tokens. We observe that finest-scale source features cannot
effectively guide the prediction of coarser target features. To bridge this
gap, we introduce a Scale-Aligned Reference (SAR) module, which injects
scale-matched conditioning information into the first self-attention layer.
VAREdit demonstrates significant advancements in both editing adherence and
efficiency. On standard benchmarks, it outperforms leading diffusion-based
methods by 30\%+ higher GPT-Balance score. Moreover, it completes a
512times512 editing in 1.2 seconds, making it 2.2times faster than the
similarly sized UltraEdit. The models are available at
https://github.com/HiDream-ai/VAREdit.
VAREdit, a visual autoregressive framework, improves image editing adherence and efficiency by using a Scale-Aligned Reference module to condition source image tokens.
diffusion models visual autoregressive next-scale prediction source image features text instructions multi-scale target features self-attention layer Scale-Aligned Reference GPT-Balance score UltraEdit