Make Pixels Dance: High-Dynamic Video Generation
Creating high-dynamic videos such as motion-rich actions and sophisticated
visual effects poses a significant challenge in the field of artificial
intelligence. Unfortunately, current state-of-the-art video generation methods,
primarily focusing on text-to-video generation, tend to produce video clips
with minimal motions despite maintaining high fidelity. We argue that relying
solely on text instructions is insufficient and suboptimal for video
generation. In this paper, we introduce PixelDance, a novel approach based on
diffusion models that incorporates image instructions for both the first and
last frames in conjunction with text instructions for video generation.
Comprehensive experimental results demonstrate that PixelDance trained with
public data exhibits significantly better proficiency in synthesizing videos
with complex scenes and intricate motions, setting a new standard for video
generation.
655c34faf63eaf2a751b2bda
Yan Zeng
admin_assigned
2023-11-21T10:27:02.859Z
640ef2b2f2d7c41a1e9b6013
/avatars/556263c07df93942c22cda64e65b2e41.svg
Yan Zeng
user
zengyan-97
655c34faf63eaf2a751b2bdb
Guoqiang Wei
admin_assigned
2023-11-21T10:27:09.416Z
63160a1d29411a6864b0e585
https://cdn-avatars.huggingface.co/v1/production/uploads/1668083049513-63160a1d29411a6864b0e585.jpeg
Guoqiang Wei
user
weigq
655c34faf63eaf2a751b2bdc
Jiani Zheng
claimed_verified
2023-11-21T15:00:50.619Z
64531f631a57e1179c203e6b
https://cdn-avatars.huggingface.co/v1/production/uploads/64531f631a57e1179c203e6b/C_J7pXFLqoJoHYPPhK3J9.jpeg
zjn
user
garlicisnotmyfavor
655c34faf63eaf2a751b2bdd
Jiaxin Zou
claimed_verified
2023-11-21T13:04:54.854Z
655c485e71ed242b42701fb0
https://cdn-avatars.huggingface.co/v1/production/uploads/655c485e71ed242b42701fb0/7d6lHB-wMy7xNj3dWIp5y.jpeg
Jiaxin Zou
user
shayeboshi
655c34faf63eaf2a751b2bde
Yang Wei
admin_assigned
2023-11-21T10:47:43.045Z
631a089f223d36c5706c7ba3
/avatars/a2b8b303d5d912b531130be44b82f601.svg
Yang Wei
user
godweiyang
655c34faf63eaf2a751b2bdf
Yuchen Zhang
655c34faf63eaf2a751b2be0
Hang Li
Make Pixels Dance: High-Dynamic Video Generation
Creating high-dynamic videos such as motion-rich actions and sophisticated
visual effects poses a significant challenge in the field of artificial
intelligence. Unfortunately, current state-of-the-art video generation methods,
primarily focusing on text-to-video generation, tend to produce video clips
with minimal motions despite maintaining high fidelity. We argue that relying
solely on text instructions is insufficient and suboptimal for video
generation. In this paper, we introduce PixelDance, a novel approach based on
diffusion models that incorporates image instructions for both the first and
last frames in conjunction with text instructions for video generation.
Comprehensive experimental results demonstrate that PixelDance trained with
public data exhibits significantly better proficiency in synthesizing videos
with complex scenes and intricate motions, setting a new standard for video
generation.
PixelDance, a diffusion model-based approach, generates high-dynamic videos by incorporating image instructions for first and last frames alongside text instructions, surpassing current text-to-video methods in complexity and motion.
diffusion models image instructions text instructions video generation complex scenes intricate motions