GeometryCrafter: Consistent Geometry Estimation for Open-world Videos
  with Diffusion Priors
Despite remarkable advancements in video depth estimation, existing methods
exhibit inherent limitations in achieving geometric fidelity through the
affine-invariant predictions, limiting their applicability in reconstruction
and other metrically grounded downstream tasks. We propose GeometryCrafter, a
novel framework that recovers high-fidelity point map sequences with temporal
coherence from open-world videos, enabling accurate 3D/4D reconstruction,
camera parameter estimation, and other depth-based applications. At the core of
our approach lies a point map Variational Autoencoder (VAE) that learns a
latent space agnostic to video latent distributions for effective point map
encoding and decoding. Leveraging the VAE, we train a video diffusion model to
model the distribution of point map sequences conditioned on the input videos.
Extensive evaluations on diverse datasets demonstrate that GeometryCrafter
achieves state-of-the-art 3D accuracy, temporal consistency, and generalization
capability.
67ec958ebb1d6dd924f94a31
Tian-Xing Xu
admin_assigned
2025-04-02T09:51:06.226Z
65f8e4778dc7bb5b4db97f92
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/gBmQdovmANmjV72k3gaW8.png
Tian-Xing Xu
user
slothfulxtx
67ec958ebb1d6dd924f94a32
Xiangjun Gao
admin_assigned
2025-04-02T09:51:14.978Z
64c0953a8137192a1e2474dc
/avatars/546405a7eaf2f60ad108ceaa0dda7d08.svg
xiangjun gao
user
xiangjun0211
67ec958ebb1d6dd924f94a33
Wenbo Hu
claimed_verified
2025-04-02T08:22:52.313Z
657a7458afbb0117ba15c59f
https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
Wenbo Hu
user
wbhu-tc
67ec958ebb1d6dd924f94a34
Xiaoyu Li
67ec958ebb1d6dd924f94a35
Song-Hai Zhang
67ec958ebb1d6dd924f94a36
Ying Shan
admin_assigned
2025-04-02T09:51:45.869Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
GeometryCrafter: Consistent Geometry Estimation for Open-world Videos
  with Diffusion Priors
Despite remarkable advancements in video depth estimation, existing methods
exhibit inherent limitations in achieving geometric fidelity through the
affine-invariant predictions, limiting their applicability in reconstruction
and other metrically grounded downstream tasks. We propose GeometryCrafter, a
novel framework that recovers high-fidelity point map sequences with temporal
coherence from open-world videos, enabling accurate 3D/4D reconstruction,
camera parameter estimation, and other depth-based applications. At the core of
our approach lies a point map Variational Autoencoder (VAE) that learns a
latent space agnostic to video latent distributions for effective point map
encoding and decoding. Leveraging the VAE, we train a video diffusion model to
model the distribution of point map sequences conditioned on the input videos.
Extensive evaluations on diverse datasets demonstrate that GeometryCrafter
achieves state-of-the-art 3D accuracy, temporal consistency, and generalization
capability.
GeometryCrafter point map Variational Autoencoder (VAE) latent space video latent distributions point map encoding point map decoding video diffusion model point map sequences 3D accuracy temporal consistency generalization capability