CameraCtrl: Enabling Camera Control for Text-to-Video Generation
Controllability plays a crucial role in video generation since it allows
users to create desired content. However, existing models largely overlooked
the precise control of camera pose that serves as a cinematic language to
express deeper narrative nuances. To alleviate this issue, we introduce
CameraCtrl, enabling accurate camera pose control for text-to-video(T2V)
models. After precisely parameterizing the camera trajectory, a plug-and-play
camera module is then trained on a T2V model, leaving others untouched.
Additionally, a comprehensive study on the effect of various datasets is also
conducted, suggesting that videos with diverse camera distribution and similar
appearances indeed enhance controllability and generalization. Experimental
results demonstrate the effectiveness of CameraCtrl in achieving precise and
domain-adaptive camera control, marking a step forward in the pursuit of
dynamic and customized video storytelling from textual and camera pose inputs.
Our project website is at: https://hehao13.github.io/projects-CameraCtrl/.
660cb980062846c7a884bf2e
Hao He
660cb980062846c7a884bf2f
Yinghao Xu
660cb980062846c7a884bf30
Yuwei Guo
claimed_verified
2025-03-16T21:24:07.347Z
6371f83d5ffa922d638ef486
https://cdn-avatars.huggingface.co/v1/production/uploads/6371f83d5ffa922d638ef486/UzWl3xhVvtol3Rz1mjQ9t.jpeg
Yuwei Guo
user
guoyww
660cb980062846c7a884bf31
Gordon Wetzstein
660cb980062846c7a884bf32
Bo Dai
660cb980062846c7a884bf33
Hongsheng Li
660cb980062846c7a884bf34
Ceyuan Yang
CameraCtrl: Enabling Camera Control for Text-to-Video Generation
Controllability plays a crucial role in video generation since it allows
users to create desired content. However, existing models largely overlooked
the precise control of camera pose that serves as a cinematic language to
express deeper narrative nuances. To alleviate this issue, we introduce
CameraCtrl, enabling accurate camera pose control for text-to-video(T2V)
models. After precisely parameterizing the camera trajectory, a plug-and-play
camera module is then trained on a T2V model, leaving others untouched.
Additionally, a comprehensive study on the effect of various datasets is also
conducted, suggesting that videos with diverse camera distribution and similar
appearances indeed enhance controllability and generalization. Experimental
results demonstrate the effectiveness of CameraCtrl in achieving precise and
domain-adaptive camera control, marking a step forward in the pursuit of
dynamic and customized video storytelling from textual and camera pose inputs.
Our project website is at: https://hehao13.github.io/projects-CameraCtrl/.
CameraCtrl enhances video generation by enabling precise camera pose control for text-to-video models, improving controllability and generalization with diverse datasets.
camera pose control text-to-video camera module dataset controllability generalization video generation