MagicMotion: Controllable Video Generation with Dense-to-Sparse
  Trajectory Guidance
Recent advances in video generation have led to remarkable improvements in
visual quality and temporal coherence. Upon this, trajectory-controllable video
generation has emerged to enable precise object motion control through
explicitly defined spatial paths. However, existing methods struggle with
complex object movements and multi-object motion control, resulting in
imprecise trajectory adherence, poor object consistency, and compromised visual
quality. Furthermore, these methods only support trajectory control in a single
format, limiting their applicability in diverse scenarios. Additionally, there
is no publicly available dataset or benchmark specifically tailored for
trajectory-controllable video generation, hindering robust training and
systematic evaluation. To address these challenges, we introduce MagicMotion, a
novel image-to-video generation framework that enables trajectory control
through three levels of conditions from dense to sparse: masks, bounding boxes,
and sparse boxes. Given an input image and trajectories, MagicMotion seamlessly
animates objects along defined trajectories while maintaining object
consistency and visual quality. Furthermore, we present MagicData, a
large-scale trajectory-controlled video dataset, along with an automated
pipeline for annotation and filtering. We also introduce MagicBench, a
comprehensive benchmark that assesses both video quality and trajectory control
accuracy across different numbers of objects. Extensive experiments demonstrate
that MagicMotion outperforms previous methods across various metrics. Our
project page are publicly available at
https://quanhaol.github.io/magicmotion-site.
67dcd5913713a0e1da19bbe5
Quanhao Li
claimed_verified
2025-03-21T11:40:57.203Z
65818917554131ec95df3829
https://cdn-avatars.huggingface.co/v1/production/uploads/65818917554131ec95df3829/YIbgLCllm-0u88pRLhpVZ.jpeg
Quanhao Li
user
quanhaol
67dcd5913713a0e1da19bbe6
Zhen Xing
67dcd5913713a0e1da19bbe7
Rui Wang
67dcd5913713a0e1da19bbe8
Hui Zhang
67dcd5913713a0e1da19bbe9
Qi Dai
67dcd5913713a0e1da19bbea
Zuxuan Wu
MagicMotion: Controllable Video Generation with Dense-to-Sparse
  Trajectory Guidance
Recent advances in video generation have led to remarkable improvements in
visual quality and temporal coherence. Upon this, trajectory-controllable video
generation has emerged to enable precise object motion control through
explicitly defined spatial paths. However, existing methods struggle with
complex object movements and multi-object motion control, resulting in
imprecise trajectory adherence, poor object consistency, and compromised visual
quality. Furthermore, these methods only support trajectory control in a single
format, limiting their applicability in diverse scenarios. Additionally, there
is no publicly available dataset or benchmark specifically tailored for
trajectory-controllable video generation, hindering robust training and
systematic evaluation. To address these challenges, we introduce MagicMotion, a
novel image-to-video generation framework that enables trajectory control
through three levels of conditions from dense to sparse: masks, bounding boxes,
and sparse boxes. Given an input image and trajectories, MagicMotion seamlessly
animates objects along defined trajectories while maintaining object
consistency and visual quality. Furthermore, we present MagicData, a
large-scale trajectory-controlled video dataset, along with an automated
pipeline for annotation and filtering. We also introduce MagicBench, a
comprehensive benchmark that assesses both video quality and trajectory control
accuracy across different numbers of objects. Extensive experiments demonstrate
that MagicMotion outperforms previous methods across various metrics. Our
project page are publicly available at
https://quanhaol.github.io/magicmotion-site.
MagicMotion is a trajectory-controllable video generation framework that supports dense, bounding box, and sparse box conditions, improving object consistency and visual quality, and is validated using the MagicData dataset and MagicBench benchmark.
trajectory-controllable video generation masks bounding boxes sparse boxes object consistency visual quality large-scale trajectory-controlled video dataset automated pipeline annotation filtering comprehensive benchmark video quality trajectory control accuracy