DragVideo: Interactive Drag-style Video Editing
Editing visual content on videos remains a formidable challenge with two main
issues: 1) direct and easy user control to produce 2) natural editing results
without unsightly distortion and artifacts after changing shape, expression and
layout. Inspired by DragGAN, a recent image-based drag-style editing technique,
we address above issues by proposing DragVideo, where a similar drag-style user
interaction is adopted to edit video content while maintaining temporal
consistency. Empowered by recent diffusion models as in DragDiffusion,
DragVideo contains the novel Drag-on-Video U-Net (DoVe) editing method, which
optimizes diffused video latents generated by video U-Net to achieve the
desired control. Specifically, we use Sample-specific LoRA fine-tuning and
Mutual Self-Attention control to ensure faithful reconstruction of video from
the DoVe method. We also present a series of testing examples for drag-style
video editing and conduct extensive experiments across a wide array of
challenging editing tasks, such as motion editing, skeleton editing, etc,
underscoring DragVideo's versatility and generality. Our codes including the
DragVideo web user interface will be released.
656fe5ae34440bdf2eda5335
Yufan Deng
claimed_verified
2023-12-06T13:27:12.313Z
657073d2642d7f0ac42e3d6c
/avatars/57cd9f4ca5590318042be8a0050485d7.svg
Yufan Deng
user
yfdeng
656fe5ae34440bdf2eda5336
Ruida Wang
claimed_verified
2024-07-10T15:12:47.192Z
6618d1c3c1167c8d8702b19d
/avatars/24611ca6c4158f4978ac8476a87d8d9c.svg
Ruida WANG
user
RickyDeSkywalker
656fe5ae34440bdf2eda5337
Yuhao Zhang
claimed_verified
2023-12-06T12:44:47.502Z
640ada00b871d7117d5ef66f
/avatars/b45f369a370102ee338384c2154e41be.svg
ZHANG Yuaho
user
yzhanglp
656fe5ae34440bdf2eda5338
Yu-Wing Tai
656fe5ae34440bdf2eda5339
Chi-Keung Tang
DragVideo: Interactive Drag-style Video Editing
Editing visual content on videos remains a formidable challenge with two main
issues: 1) direct and easy user control to produce 2) natural editing results
without unsightly distortion and artifacts after changing shape, expression and
layout. Inspired by DragGAN, a recent image-based drag-style editing technique,
we address above issues by proposing DragVideo, where a similar drag-style user
interaction is adopted to edit video content while maintaining temporal
consistency. Empowered by recent diffusion models as in DragDiffusion,
DragVideo contains the novel Drag-on-Video U-Net (DoVe) editing method, which
optimizes diffused video latents generated by video U-Net to achieve the
desired control. Specifically, we use Sample-specific LoRA fine-tuning and
Mutual Self-Attention control to ensure faithful reconstruction of video from
the DoVe method. We also present a series of testing examples for drag-style
video editing and conduct extensive experiments across a wide array of
challenging editing tasks, such as motion editing, skeleton editing, etc,
underscoring DragVideo's versatility and generality. Our codes including the
DragVideo web user interface will be released.
DragVideo, a drag-style video editing technique, uses diffusion models and U-Net architecture for natural and user-controlled editing while maintaining temporal consistency and versatility.
DragGAN drag-style editing diffusion models DragDiffusion Drag-on-Video U-Net DoVe Sample-specific LoRA fine-tuning Mutual Self-Attention control motion editing skeleton editing