GenDoP: Auto-regressive Camera Trajectory Generation as a Director of
  Photography
Camera trajectory design plays a crucial role in video production, serving as
a fundamental tool for conveying directorial intent and enhancing visual
storytelling. In cinematography, Directors of Photography meticulously craft
camera movements to achieve expressive and intentional framing. However,
existing methods for camera trajectory generation remain limited: Traditional
approaches rely on geometric optimization or handcrafted procedural systems,
while recent learning-based methods often inherit structural biases or lack
textual alignment, constraining creative synthesis. In this work, we introduce
an auto-regressive model inspired by the expertise of Directors of Photography
to generate artistic and expressive camera trajectories. We first introduce
DataDoP, a large-scale multi-modal dataset containing 29K real-world shots with
free-moving camera trajectories, depth maps, and detailed captions in specific
movements, interaction with the scene, and directorial intent. Thanks to the
comprehensive and diverse database, we further train an auto-regressive,
decoder-only Transformer for high-quality, context-aware camera movement
generation based on text guidance and RGBD inputs, named GenDoP. Extensive
experiments demonstrate that compared to existing methods, GenDoP offers better
controllability, finer-grained trajectory adjustments, and higher motion
stability. We believe our approach establishes a new standard for
learning-based cinematography, paving the way for future advancements in camera
control and filmmaking. Our project website:
https://kszpxxzmc.github.io/GenDoP/.
67f72c452eec6ce5c8b9e8e6
Mengchen Zhang
claimed_verified
2025-04-10T06:39:42.813Z
64de20c5808492ba6e65d124
https://cdn-avatars.huggingface.co/v1/production/uploads/64de20c5808492ba6e65d124/58IX_TI5vJw73qS1knw56.jpeg
Zhang Mengchen
user
Dubhe-zmc
67f72c452eec6ce5c8b9e8e7
Tong Wu
67f72c452eec6ce5c8b9e8e8
Jing Tan
admin_assigned
2025-04-10T08:01:54.122Z
65367c40061949598892dbdc
/avatars/4baf27263841471cbd5f629a8b99424d.svg
Jing Tan
user
jingtan
67f72c452eec6ce5c8b9e8e9
Ziwei Liu
admin_assigned
2025-04-10T08:01:11.166Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
67f72c452eec6ce5c8b9e8ea
Gordon Wetzstein
admin_assigned
2025-04-10T08:01:03.935Z
6694e583ac96ca2c17131505
/avatars/6e7a31f257e36cf301da6f879dc0a122.svg
Gordon Wetzstein
user
wetzste1
67f72c452eec6ce5c8b9e8eb
Dahua Lin
admin_assigned
2025-04-10T08:00:57.092Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
GenDoP: Auto-regressive Camera Trajectory Generation as a Director of
  Photography
Camera trajectory design plays a crucial role in video production, serving as
a fundamental tool for conveying directorial intent and enhancing visual
storytelling. In cinematography, Directors of Photography meticulously craft
camera movements to achieve expressive and intentional framing. However,
existing methods for camera trajectory generation remain limited: Traditional
approaches rely on geometric optimization or handcrafted procedural systems,
while recent learning-based methods often inherit structural biases or lack
textual alignment, constraining creative synthesis. In this work, we introduce
an auto-regressive model inspired by the expertise of Directors of Photography
to generate artistic and expressive camera trajectories. We first introduce
DataDoP, a large-scale multi-modal dataset containing 29K real-world shots with
free-moving camera trajectories, depth maps, and detailed captions in specific
movements, interaction with the scene, and directorial intent. Thanks to the
comprehensive and diverse database, we further train an auto-regressive,
decoder-only Transformer for high-quality, context-aware camera movement
generation based on text guidance and RGBD inputs, named GenDoP. Extensive
experiments demonstrate that compared to existing methods, GenDoP offers better
controllability, finer-grained trajectory adjustments, and higher motion
stability. We believe our approach establishes a new standard for
learning-based cinematography, paving the way for future advancements in camera
control and filmmaking. Our project website:
https://kszpxxzmc.github.io/GenDoP/.
A novel auto-regressive Transformer model, GenDoP, generates high-quality, text-guided, and context-aware camera trajectories using a large-scale multi-modal dataset, improving controllability and motion stability in cinematography.
DataDoP decoder-only Transformer auto-regressive model camera trajectory generation text guidance RGBD inputs motion stability controllability trajectory adjustments directorial intent