VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video
  Diffusion Models
In this paper, we propose VideoFrom3D, a novel framework for synthesizing
high-quality 3D scene videos from coarse geometry, a camera trajectory, and a
reference image. Our approach streamlines the 3D graphic design workflow,
enabling flexible design exploration and rapid production of deliverables. A
straightforward approach to synthesizing a video from coarse geometry might
condition a video diffusion model on geometric structure. However, existing
video diffusion models struggle to generate high-fidelity results for complex
scenes due to the difficulty of jointly modeling visual quality, motion, and
temporal consistency. To address this, we propose a generative framework that
leverages the complementary strengths of image and video diffusion models.
Specifically, our framework consists of a Sparse Anchor-view Generation (SAG)
and a Geometry-guided Generative Inbetweening (GGI) module. The SAG module
generates high-quality, cross-view consistent anchor views using an image
diffusion model, aided by Sparse Appearance-guided Sampling. Building on these
anchor views, GGI module faithfully interpolates intermediate frames using a
video diffusion model, enhanced by flow-based camera control and structural
guidance. Notably, both modules operate without any paired dataset of 3D scene
models and natural images, which is extremely difficult to obtain.
Comprehensive experiments show that our method produces high-quality,
style-consistent scene videos under diverse and challenging scenarios,
outperforming simple and extended baselines.
68d21e5d1ca7156988a8edc5
Geonung Kim
admin_assigned
2025-09-23T14:15:01.645Z
63040f5c7373aacccd889430
/avatars/29b05c69445c48943f535ad381fb9464.svg
geonung kim
user
comar
68d21e5d1ca7156988a8edc6
Janghyeok Han
admin_assigned
2025-09-23T14:15:09.662Z
642cec5dfc341371b030adaa
/avatars/13886df0cf19d697d181a0691ac269fc.svg
Janghyeok Han
user
Janghyeok
68d21e5d1ca7156988a8edc7
Sunghyun Cho
admin_assigned
2025-09-23T14:15:17.945Z
675d3d94036615f450fe818f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Oc4-UMIHyVe0naZ0XGQmj.png
sunghyun cho
user
chosh1110
VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video
  Diffusion Models
In this paper, we propose VideoFrom3D, a novel framework for synthesizing
high-quality 3D scene videos from coarse geometry, a camera trajectory, and a
reference image. Our approach streamlines the 3D graphic design workflow,
enabling flexible design exploration and rapid production of deliverables. A
straightforward approach to synthesizing a video from coarse geometry might
condition a video diffusion model on geometric structure. However, existing
video diffusion models struggle to generate high-fidelity results for complex
scenes due to the difficulty of jointly modeling visual quality, motion, and
temporal consistency. To address this, we propose a generative framework that
leverages the complementary strengths of image and video diffusion models.
Specifically, our framework consists of a Sparse Anchor-view Generation (SAG)
and a Geometry-guided Generative Inbetweening (GGI) module. The SAG module
generates high-quality, cross-view consistent anchor views using an image
diffusion model, aided by Sparse Appearance-guided Sampling. Building on these
anchor views, GGI module faithfully interpolates intermediate frames using a
video diffusion model, enhanced by flow-based camera control and structural
guidance. Notably, both modules operate without any paired dataset of 3D scene
models and natural images, which is extremely difficult to obtain.
Comprehensive experiments show that our method produces high-quality,
style-consistent scene videos under diverse and challenging scenarios,
outperforming simple and extended baselines.
VideoFrom3D synthesizes high-quality 3D scene videos using a combination of image and video diffusion models, achieving style consistency without requiring paired datasets.
video diffusion model image diffusion model Sparse Anchor-view Generation Geometry-guided Generative Inbetweening Sparse Appearance-guided Sampling flow-based camera control structural guidance