Gen-L-Video: Multi-Text to Long Video Generation via Temporal
  Co-Denoising
Leveraging large-scale image-text datasets and advancements in diffusion
models, text-driven generative models have made remarkable strides in the field
of image generation and editing. This study explores the potential of extending
the text-driven ability to the generation and editing of multi-text conditioned
long videos. Current methodologies for video generation and editing, while
innovative, are often confined to extremely short videos (typically less than
24 frames) and are limited to a single text condition. These constraints
significantly limit their applications given that real-world videos usually
consist of multiple segments, each bearing different semantic information. To
address this challenge, we introduce a novel paradigm dubbed as Gen-L-Video,
capable of extending off-the-shelf short video diffusion models for generating
and editing videos comprising hundreds of frames with diverse semantic segments
without introducing additional training, all while preserving content
consistency. We have implemented three mainstream text-driven video generation
and editing methodologies and extended them to accommodate longer videos imbued
with a variety of semantic segments with our proposed paradigm. Our
experimental outcomes reveal that our approach significantly broadens the
generative and editing capabilities of video diffusion models, offering new
possibilities for future research and applications. The code is available at
https://github.com/G-U-N/Gen-L-Video.
64757dd2a51711a3b591ad3d
Fu-Yun Wang
claimed_verified
2023-05-30T09:01:41.902Z
63e9e92f20c109718713f5eb
/avatars/9ff312e854d803e1a2e9e685a21d12f8.svg
Fu-Yun Wang
user
wangfuyun
64757dd2a51711a3b591ad3e
Wenshuo Chen
claimed_verified
2023-05-30T10:08:06.881Z
6475be9937ed88a749e63cb6
/avatars/37df06697e8c4638e6f0ad49470dc4e3.svg
Wenshuo Chen
user
winshotchen
64757dd2a51711a3b591ad3f
Guanglu Song
64757dd2a51711a3b591ad40
Han-Jia Ye
64757dd2a51711a3b591ad41
Yu Liu
64757dd2a51711a3b591ad42
Hongsheng Li
claimed_verified
2024-02-05T10:51:10.529Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
Gen-L-Video: Multi-Text to Long Video Generation via Temporal
  Co-Denoising
Leveraging large-scale image-text datasets and advancements in diffusion
models, text-driven generative models have made remarkable strides in the field
of image generation and editing. This study explores the potential of extending
the text-driven ability to the generation and editing of multi-text conditioned
long videos. Current methodologies for video generation and editing, while
innovative, are often confined to extremely short videos (typically less than
24 frames) and are limited to a single text condition. These constraints
significantly limit their applications given that real-world videos usually
consist of multiple segments, each bearing different semantic information. To
address this challenge, we introduce a novel paradigm dubbed as Gen-L-Video,
capable of extending off-the-shelf short video diffusion models for generating
and editing videos comprising hundreds of frames with diverse semantic segments
without introducing additional training, all while preserving content
consistency. We have implemented three mainstream text-driven video generation
and editing methodologies and extended them to accommodate longer videos imbued
with a variety of semantic segments with our proposed paradigm. Our
experimental outcomes reveal that our approach significantly broadens the
generative and editing capabilities of video diffusion models, offering new
possibilities for future research and applications. The code is available at
https://github.com/G-U-N/Gen-L-Video.
A novel paradigm extends short video diffusion models to handle long, multi-segment, text-conditioned video generation and editing without additional training.
diffusion models text-driven generative models image generation long videos semantic segments content consistency Gen-L-Video text-driven video generation text-driven video editing