DiffuEraser: A Diffusion Model for Video Inpainting
Recent video inpainting algorithms integrate flow-based pixel propagation
with transformer-based generation to leverage optical flow for restoring
textures and objects using information from neighboring frames, while
completing masked regions through visual Transformers. However, these
approaches often encounter blurring and temporal inconsistencies when dealing
with large masks, highlighting the need for models with enhanced generative
capabilities. Recently, diffusion models have emerged as a prominent technique
in image and video generation due to their impressive performance. In this
paper, we introduce DiffuEraser, a video inpainting model based on stable
diffusion, designed to fill masked regions with greater details and more
coherent structures. We incorporate prior information to provide initialization
and weak conditioning,which helps mitigate noisy artifacts and suppress
hallucinations. Additionally, to improve temporal consistency during
long-sequence inference, we expand the temporal receptive fields of both the
prior model and DiffuEraser, and further enhance consistency by leveraging the
temporal smoothing property of Video Diffusion Models. Experimental results
demonstrate that our proposed method outperforms state-of-the-art techniques in
both content completeness and temporal consistency while maintaining acceptable
efficiency.
678e125d09dc6d3a311cc04e
Xiaowen Li
admin_assigned
2025-01-24T10:31:02.655Z
6497b4464a3c31df8e4148d8
/avatars/4397a380468e84bc7945fddd9a6d1066.svg
Xiaowen Li
user
asLKHFksasak
678e125d09dc6d3a311cc04f
Haolan Xue
678e125d09dc6d3a311cc050
Peiran Ren
admin_assigned
2025-01-24T10:30:47.443Z
64b74a45f902508f0d786505
/avatars/8bc5aaa011642827e12524c4f0a56927.svg
Peiran REN
user
lyraestar
678e125d09dc6d3a311cc051
Liefeng Bo
admin_assigned
2025-01-24T10:30:55.550Z
63d0cc736b985b0f25d0412c
/avatars/3eb8c79f9a7c4c819038ea7b04e323dd.svg
Bo
user
Liefeng
DiffuEraser: A Diffusion Model for Video Inpainting
Recent video inpainting algorithms integrate flow-based pixel propagation
with transformer-based generation to leverage optical flow for restoring
textures and objects using information from neighboring frames, while
completing masked regions through visual Transformers. However, these
approaches often encounter blurring and temporal inconsistencies when dealing
with large masks, highlighting the need for models with enhanced generative
capabilities. Recently, diffusion models have emerged as a prominent technique
in image and video generation due to their impressive performance. In this
paper, we introduce DiffuEraser, a video inpainting model based on stable
diffusion, designed to fill masked regions with greater details and more
coherent structures. We incorporate prior information to provide initialization
and weak conditioning,which helps mitigate noisy artifacts and suppress
hallucinations. Additionally, to improve temporal consistency during
long-sequence inference, we expand the temporal receptive fields of both the
prior model and DiffuEraser, and further enhance consistency by leveraging the
temporal smoothing property of Video Diffusion Models. Experimental results
demonstrate that our proposed method outperforms state-of-the-art techniques in
both content completeness and temporal consistency while maintaining acceptable
efficiency.
DiffuEraser, a video inpainting model based on stable diffusion with enhanced generative and temporal consistency capabilities, outperforms state-of-the-art techniques.
flow-based pixel propagation transformer-based generation optical flow diffusion models stable diffusion visual Transformers prior information weak conditioning noisy artifacts hallucinations temporal consistency temporal receptive fields Video Diffusion Models long-sequence inference temporal smoothing property