I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion
  Models
The remarkable generative capabilities of diffusion models have motivated
extensive research in both image and video editing. Compared to video editing
which faces additional challenges in the time dimension, image editing has
witnessed the development of more diverse, high-quality approaches and more
capable software like Photoshop. In light of this gap, we introduce a novel and
generic solution that extends the applicability of image editing tools to
videos by propagating edits from a single frame to the entire video using a
pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively
preserves the visual and motion integrity of the source video depending on the
extent of the edits, effectively handling global edits, local edits, and
moderate shape changes, which existing methods cannot fully achieve. At the
core of our method are two main processes: Coarse Motion Extraction to align
basic motion patterns with the original video, and Appearance Refinement for
precise adjustments using fine-grained attention matching. We also incorporate
a skip-interval strategy to mitigate quality degradation from auto-regressive
generation across multiple video clips. Experimental results demonstrate our
framework's superior performance in fine-grained video editing, proving its
capability to produce high-quality, temporally consistent outputs.
66555d9ca2d7a882a87cc9b0
Wenqi Ouyang
admin_assigned
2024-05-28T10:31:30.172Z
6487d9415d164d8c7f905bcf
/avatars/1c01dd4b88f88a9863a12c09a9a28bed.svg
Wenqi Ouyang
user
Vicky0522
66555d9ca2d7a882a87cc9b1
Yi Dong
claimed_verified
2024-05-28T15:21:39.190Z
661f639f7273a6cb22767c14
/avatars/8b65440edf21e1fd98c90e2776bd8e0c.svg
Dong Yi
user
ntuyi
66555d9ca2d7a882a87cc9b2
Lei Yang
admin_assigned
2025-03-10T08:09:53.918Z
6626a471430a124253f197c8
/avatars/f5747fdbe495d1296fed9d16d8c95857.svg
yl-1993
user
yl-1993
66555d9ca2d7a882a87cc9b3
Jianlou Si
admin_assigned
2024-05-28T10:30:41.704Z
6409693abf3e9a4bb2e64dfd
/avatars/9ddeceec9f4dae3a813165016eb6f4d3.svg
Jianlou
user
Jianlou
66555d9ca2d7a882a87cc9b4
Xingang Pan
admin_assigned
2024-05-28T10:30:21.693Z
646758d6a9b4610868a138ff
https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
Xingang Pan
user
Xingang-Pan
I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion
  Models
The remarkable generative capabilities of diffusion models have motivated
extensive research in both image and video editing. Compared to video editing
which faces additional challenges in the time dimension, image editing has
witnessed the development of more diverse, high-quality approaches and more
capable software like Photoshop. In light of this gap, we introduce a novel and
generic solution that extends the applicability of image editing tools to
videos by propagating edits from a single frame to the entire video using a
pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively
preserves the visual and motion integrity of the source video depending on the
extent of the edits, effectively handling global edits, local edits, and
moderate shape changes, which existing methods cannot fully achieve. At the
core of our method are two main processes: Coarse Motion Extraction to align
basic motion patterns with the original video, and Appearance Refinement for
precise adjustments using fine-grained attention matching. We also incorporate
a skip-interval strategy to mitigate quality degradation from auto-regressive
generation across multiple video clips. Experimental results demonstrate our
framework's superior performance in fine-grained video editing, proving its
capability to produce high-quality, temporally consistent outputs.
A novel method extends image editing to videos by aligning motion and refining appearance, maintaining visual and motion integrity with high temporal consistency.
diffusion models image-to-video model Coarse Motion Extraction Appearance Refinement fine-grained attention matching skip-interval strategy auto-regressive generation