Direct-a-Video: Customized Video Generation with User-Directed Camera
  Movement and Object Motion
Recent text-to-video diffusion models have achieved impressive progress. In
practice, users often desire the ability to control object motion and camera
movement independently for customized video creation. However, current methods
lack the focus on separately controlling object motion and camera movement in a
decoupled manner, which limits the controllability and flexibility of
text-to-video models. In this paper, we introduce Direct-a-Video, a system that
allows users to independently specify motions for one or multiple objects
and/or camera movements, as if directing a video. We propose a simple yet
effective strategy for the decoupled control of object motion and camera
movement. Object motion is controlled through spatial cross-attention
modulation using the model's inherent priors, requiring no additional
optimization. For camera movement, we introduce new temporal cross-attention
layers to interpret quantitative camera movement parameters. We further employ
an augmentation-based approach to train these layers in a self-supervised
manner on a small-scale dataset, eliminating the need for explicit motion
annotation. Both components operate independently, allowing individual or
combined control, and can generalize to open-domain scenarios. Extensive
experiments demonstrate the superiority and effectiveness of our method.
Project page: https://direct-a-video.github.io/.
65c1a29532ff878c506343a7
Shiyuan Yang
claimed_verified
2025-03-17T14:43:36.241Z
63316d499e3604f3f17f5d89
/avatars/c7e0c8bdd7e598276cfd133d5222c5e8.svg
catfood
user
ysy31415926
65c1a29532ff878c506343a8
Liang Hou
claimed_verified
2025-07-01T10:17:06.534Z
64560a2aaaaf85a98fa9a4b9
https://cdn-avatars.huggingface.co/v1/production/uploads/64560a2aaaaf85a98fa9a4b9/2Kp0S0sMVpKqo81s-l_Yt.png
Liang Hou
user
lianghou
65c1a29532ff878c506343a9
Haibin Huang
65c1a29532ff878c506343aa
Chongyang Ma
65c1a29532ff878c506343ab
Pengfei Wan
65c1a29532ff878c506343ac
Di Zhang
65c1a29532ff878c506343ad
Xiaodong Chen
65c1a29532ff878c506343ae
Jing Liao
Direct-a-Video: Customized Video Generation with User-Directed Camera
  Movement and Object Motion
Recent text-to-video diffusion models have achieved impressive progress. In
practice, users often desire the ability to control object motion and camera
movement independently for customized video creation. However, current methods
lack the focus on separately controlling object motion and camera movement in a
decoupled manner, which limits the controllability and flexibility of
text-to-video models. In this paper, we introduce Direct-a-Video, a system that
allows users to independently specify motions for one or multiple objects
and/or camera movements, as if directing a video. We propose a simple yet
effective strategy for the decoupled control of object motion and camera
movement. Object motion is controlled through spatial cross-attention
modulation using the model's inherent priors, requiring no additional
optimization. For camera movement, we introduce new temporal cross-attention
layers to interpret quantitative camera movement parameters. We further employ
an augmentation-based approach to train these layers in a self-supervised
manner on a small-scale dataset, eliminating the need for explicit motion
annotation. Both components operate independently, allowing individual or
combined control, and can generalize to open-domain scenarios. Extensive
experiments demonstrate the superiority and effectiveness of our method.
Project page: https://direct-a-video.github.io/.
Direct-a-Video enables independent control of object motion and camera movement in text-to-video generation using spatial and temporal cross-attention modulation.
text-to-video diffusion models object motion camera movement decoupled control spatial cross-attention modulation temporal cross-attention layers self-supervised training