VideoGrain: Modulating Space-Time Attention for Multi-grained Video
  Editing
Recent advancements in diffusion models have significantly improved video
generation and editing capabilities. However, multi-grained video editing,
which encompasses class-level, instance-level, and part-level modifications,
remains a formidable challenge. The major difficulties in multi-grained editing
include semantic misalignment of text-to-region control and feature coupling
within the diffusion model. To address these difficulties, we present
VideoGrain, a zero-shot approach that modulates space-time (cross- and self-)
attention mechanisms to achieve fine-grained control over video content. We
enhance text-to-region control by amplifying each local prompt's attention to
its corresponding spatial-disentangled region while minimizing interactions
with irrelevant areas in cross-attention. Additionally, we improve feature
separation by increasing intra-region awareness and reducing inter-region
interference in self-attention. Extensive experiments demonstrate our method
achieves state-of-the-art performance in real-world scenarios. Our code, data,
and demos are available at https://knightyxp.github.io/VideoGrain_project_page/
67bd515c0417e7f92283d3b8
Xiangpeng Yang
admin_assigned
2025-03-14T08:35:35.071Z
6486df66373f79a52913e017
/avatars/4741683fbbcec3a615d0a8df62bc6fec.svg
Xiangpeng Yang
user
XiangpengYang
67bd515c0417e7f92283d3b9
Linchao Zhu
admin_assigned
2025-02-25T15:16:07.044Z
63521e1dfe367c0d9b155007
/avatars/b22804fc63b507fd60191486b17cdf7c.svg
Linchao Zhu
user
ffmpbgrnn
67bd515c0417e7f92283d3ba
Hehe Fan
admin_assigned
2025-02-25T15:16:12.884Z
64ad04020fb9b20dbabbd30e
/avatars/a6bae4a3a4bcd6b54c33860fe14c7923.svg
Hehe Fan
user
hehefan
67bd515c0417e7f92283d3bb
Yi Yang
VideoGrain: Modulating Space-Time Attention for Multi-grained Video
  Editing
Recent advancements in diffusion models have significantly improved video
generation and editing capabilities. However, multi-grained video editing,
which encompasses class-level, instance-level, and part-level modifications,
remains a formidable challenge. The major difficulties in multi-grained editing
include semantic misalignment of text-to-region control and feature coupling
within the diffusion model. To address these difficulties, we present
VideoGrain, a zero-shot approach that modulates space-time (cross- and self-)
attention mechanisms to achieve fine-grained control over video content. We
enhance text-to-region control by amplifying each local prompt's attention to
its corresponding spatial-disentangled region while minimizing interactions
with irrelevant areas in cross-attention. Additionally, we improve feature
separation by increasing intra-region awareness and reducing inter-region
interference in self-attention. Extensive experiments demonstrate our method
achieves state-of-the-art performance in real-world scenarios. Our code, data,
and demos are available at https://knightyxp.github.io/VideoGrain_project_page/
VideoGrain uses space-time attention mechanisms to enhance fine-grained video editing by improving text-to-region control and feature separation.
diffusion models multi-grained video editing text-to-region control feature coupling cross-attention self-attention spatial-disentangled region intra-region awareness inter-region interference