LumosFlow: Motion-Guided Long Video Generation
Long video generation has gained increasing attention due to its widespread
applications in fields such as entertainment and simulation. Despite advances,
synthesizing temporally coherent and visually compelling long sequences remains
a formidable challenge. Conventional approaches often synthesize long videos by
sequentially generating and concatenating short clips, or generating key frames
and then interpolate the intermediate frames in a hierarchical manner. However,
both of them still remain significant challenges, leading to issues such as
temporal repetition or unnatural transitions. In this paper, we revisit the
hierarchical long video generation pipeline and introduce LumosFlow, a
framework introduce motion guidance explicitly. Specifically, we first employ
the Large Motion Text-to-Video Diffusion Model (LMTV-DM) to generate key frames
with larger motion intervals, thereby ensuring content diversity in the
generated long videos. Given the complexity of interpolating contextual
transitions between key frames, we further decompose the intermediate frame
interpolation into motion generation and post-hoc refinement. For each pair of
key frames, the Latent Optical Flow Diffusion Model (LOF-DM) synthesizes
complex and large-motion optical flows, while MotionControlNet subsequently
refines the warped results to enhance quality and guide intermediate frame
generation. Compared with traditional video frame interpolation, we achieve 15x
interpolation, ensuring reasonable and continuous motion between adjacent
frames. Experiments show that our method can generate long videos with
consistent motion and appearance. Code and models will be made publicly
available upon acceptance. Our project page:
https://jiahaochen1.github.io/LumosFlow/
683fbc32917306517315589f
Jiahao Chen
claimed_verified
2025-06-13T07:42:09.162Z
67b534337d044bebc2e8752c
/avatars/5b086a74d966b063b72919366833c2eb.svg
Jiahao Chen
user
JiahaoChen1
683fbc3291730651731558a0
Hangjie Yuan
claimed_verified
2025-06-05T07:28:23.685Z
649d54b314afbb10ce2a9eeb
/avatars/15c325d8c2273ff63569f23015e98486.svg
Hangjie Yuan
user
JacobYuan
683fbc3291730651731558a1
Yichen Qian
683fbc3291730651731558a2
Jingyun Liang
683fbc3291730651731558a3
Jiazheng Xing
683fbc3291730651731558a4
Pengwei Liu
683fbc3291730651731558a5
Weihua Chen
683fbc3291730651731558a6
Fan Wang
683fbc3291730651731558a7
Bing Su
LumosFlow: Motion-Guided Long Video Generation
Long video generation has gained increasing attention due to its widespread
applications in fields such as entertainment and simulation. Despite advances,
synthesizing temporally coherent and visually compelling long sequences remains
a formidable challenge. Conventional approaches often synthesize long videos by
sequentially generating and concatenating short clips, or generating key frames
and then interpolate the intermediate frames in a hierarchical manner. However,
both of them still remain significant challenges, leading to issues such as
temporal repetition or unnatural transitions. In this paper, we revisit the
hierarchical long video generation pipeline and introduce LumosFlow, a
framework introduce motion guidance explicitly. Specifically, we first employ
the Large Motion Text-to-Video Diffusion Model (LMTV-DM) to generate key frames
with larger motion intervals, thereby ensuring content diversity in the
generated long videos. Given the complexity of interpolating contextual
transitions between key frames, we further decompose the intermediate frame
interpolation into motion generation and post-hoc refinement. For each pair of
key frames, the Latent Optical Flow Diffusion Model (LOF-DM) synthesizes
complex and large-motion optical flows, while MotionControlNet subsequently
refines the warped results to enhance quality and guide intermediate frame
generation. Compared with traditional video frame interpolation, we achieve 15x
interpolation, ensuring reasonable and continuous motion between adjacent
frames. Experiments show that our method can generate long videos with
consistent motion and appearance. Code and models will be made publicly
available upon acceptance. Our project page:
https://jiahaochen1.github.io/LumosFlow/
LumosFlow uses LMTV-DM for key frame generation and LOF-DM followed by MotionControlNet for smooth intermediate frame interpolation, ensuring temporally coherent long video generation.
Large Motion Text-to-Video Diffusion Model LMTV-DM Latent Optical Flow Diffusion Model LOF-DM MotionControlNet optical flows frame interpolation key frames long video generation motion guidance synthetic long videos