Frame In-N-Out: Unbounded Controllable Image-to-Video Generation
Controllability, temporal coherence, and detail synthesis remain the most
critical challenges in video generation. In this paper, we focus on a commonly
used yet underexplored cinematic technique known as Frame In and Frame Out.
Specifically, starting from image-to-video generation, users can control the
objects in the image to naturally leave the scene or provide breaking new
identity references to enter the scene, guided by user-specified motion
trajectory. To support this task, we introduce a new dataset curated
semi-automatically, a comprehensive evaluation protocol targeting this setting,
and an efficient identity-preserving motion-controllable video Diffusion
Transformer architecture. Our evaluation shows that our proposed approach
significantly outperforms existing baselines.
683672b0bf8d50a1f8ae8741
Boyang Wang
claimed_verified
2025-05-28T08:57:46.426Z
64ed876a74d9b58eabc769a4
https://cdn-avatars.huggingface.co/v1/production/uploads/64ed876a74d9b58eabc769a4/K4bJVW0FlqRtAAxJBJifR.jpeg
Boyang Wang
user
HikariDawn
683672b0bf8d50a1f8ae8742
Xuweiyi Chen
admin_assigned
2025-05-28T12:39:05.553Z
634632aaac1cb29fb2ac9f14
https://cdn-avatars.huggingface.co/v1/production/uploads/634632aaac1cb29fb2ac9f14/nGZ2TzKOOcKMAR_NFYKkR.jpeg
Xuweiyi Chen
user
Xuweiyi
683672b0bf8d50a1f8ae8743
Matheus Gadelha
admin_assigned
2025-05-28T12:39:11.640Z
6315a9c466062476ae1fa0c7
/avatars/c59749be7ecee4858c59889fea61c88a.svg
Matheus
user
matheusgadelha
683672b0bf8d50a1f8ae8744
Zezhou Cheng
admin_assigned
2025-05-28T12:39:22.963Z
64834da6c8b6f4a798f9f09d
https://cdn-avatars.huggingface.co/v1/production/uploads/64834da6c8b6f4a798f9f09d/EOxUisexBKD-L5yCD7Ecs.png
Cheng
user
Zezhou
Frame In-N-Out: Unbounded Controllable Image-to-Video Generation
Controllability, temporal coherence, and detail synthesis remain the most
critical challenges in video generation. In this paper, we focus on a commonly
used yet underexplored cinematic technique known as Frame In and Frame Out.
Specifically, starting from image-to-video generation, users can control the
objects in the image to naturally leave the scene or provide breaking new
identity references to enter the scene, guided by user-specified motion
trajectory. To support this task, we introduce a new dataset curated
semi-automatically, a comprehensive evaluation protocol targeting this setting,
and an efficient identity-preserving motion-controllable video Diffusion
Transformer architecture. Our evaluation shows that our proposed approach
significantly outperforms existing baselines.
An efficient Diffusion Transformer architecture addresses controllability, temporal coherence, and detail synthesis in video generation using the Frame In and Frame Out technique.
image-to-video generation motion trajectory video Diffusion Transformer