MotionPro: A Precise Motion Controller for Image-to-Video Generation
Animating images with interactive motion control has garnered popularity for
image-to-video (I2V) generation. Modern approaches typically rely on large
Gaussian kernels to extend motion trajectories as condition without explicitly
defining movement region, leading to coarse motion control and failing to
disentangle object and camera moving. To alleviate these, we present MotionPro,
a precise motion controller that novelly leverages region-wise trajectory and
motion mask to regulate fine-grained motion synthesis and identify target
motion category (i.e., object or camera moving), respectively. Technically,
MotionPro first estimates the flow maps on each training video via a tracking
model, and then samples the region-wise trajectories to simulate inference
scenario. Instead of extending flow through large Gaussian kernels, our
region-wise trajectory approach enables more precise control by directly
utilizing trajectories within local regions, thereby effectively characterizing
fine-grained movements. A motion mask is simultaneously derived from the
predicted flow maps to capture the holistic motion dynamics of the movement
regions. To pursue natural motion control, MotionPro further strengthens video
denoising by incorporating both region-wise trajectories and motion mask
through feature modulation. More remarkably, we meticulously construct a
benchmark, i.e., MC-Bench, with 1.1K user-annotated image-trajectory pairs, for
the evaluation of both fine-grained and object-level I2V motion control.
Extensive experiments conducted on WebVid-10M and MC-Bench demonstrate the
effectiveness of MotionPro. Please refer to our project page for more results:
https://zhw-zhang.github.io/MotionPro-page/.
6835d1587ba57211f250371d
Zhongwei Zhang
claimed_verified
2025-05-27T15:48:53.306Z
6496f5754a3c31df8e3139f6
/avatars/cf789d1986f976373c82b2976df4542a.svg
Zhongwei Zhang
user
zzwustc
6835d1587ba57211f250371e
Fuchen Long
claimed_verified
2025-05-29T07:43:53.687Z
6449f2dfeb7db8f70fb990f8
/avatars/72c02e754bb35dab05c4a5f1e69c95f1.svg
Fuchen
user
FireCRT
6835d1587ba57211f250371f
Zhaofan Qiu
6835d1587ba57211f2503720
Yingwei Pan
6835d1587ba57211f2503721
Wu Liu
6835d1587ba57211f2503722
Ting Yao
6835d1587ba57211f2503723
Tao Mei
MotionPro: A Precise Motion Controller for Image-to-Video Generation
Animating images with interactive motion control has garnered popularity for
image-to-video (I2V) generation. Modern approaches typically rely on large
Gaussian kernels to extend motion trajectories as condition without explicitly
defining movement region, leading to coarse motion control and failing to
disentangle object and camera moving. To alleviate these, we present MotionPro,
a precise motion controller that novelly leverages region-wise trajectory and
motion mask to regulate fine-grained motion synthesis and identify target
motion category (i.e., object or camera moving), respectively. Technically,
MotionPro first estimates the flow maps on each training video via a tracking
model, and then samples the region-wise trajectories to simulate inference
scenario. Instead of extending flow through large Gaussian kernels, our
region-wise trajectory approach enables more precise control by directly
utilizing trajectories within local regions, thereby effectively characterizing
fine-grained movements. A motion mask is simultaneously derived from the
predicted flow maps to capture the holistic motion dynamics of the movement
regions. To pursue natural motion control, MotionPro further strengthens video
denoising by incorporating both region-wise trajectories and motion mask
through feature modulation. More remarkably, we meticulously construct a
benchmark, i.e., MC-Bench, with 1.1K user-annotated image-trajectory pairs, for
the evaluation of both fine-grained and object-level I2V motion control.
Extensive experiments conducted on WebVid-10M and MC-Bench demonstrate the
effectiveness of MotionPro. Please refer to our project page for more results:
https://zhw-zhang.github.io/MotionPro-page/.
MotionPro uses region-wise trajectories and motion masks for precise image-to-video generation, enhancing motion control and disentangling object and camera movement.
Gaussian kernels motion control motion trajectories motion mask flow maps tracking model feature modulation fine-grained motion MC-Bench WebVid-10M video denoising