NormalCrafter: Learning Temporally Consistent Normals from Video
  Diffusion Priors
Surface normal estimation serves as a cornerstone for a spectrum of computer
vision applications. While numerous efforts have been devoted to static image
scenarios, ensuring temporal coherence in video-based normal estimation remains
a formidable challenge. Instead of merely augmenting existing methods with
temporal components, we present NormalCrafter to leverage the inherent temporal
priors of video diffusion models. To secure high-fidelity normal estimation
across sequences, we propose Semantic Feature Regularization (SFR), which
aligns diffusion features with semantic cues, encouraging the model to
concentrate on the intrinsic semantics of the scene. Moreover, we introduce a
two-stage training protocol that leverages both latent and pixel space learning
to preserve spatial accuracy while maintaining long temporal context. Extensive
evaluations demonstrate the efficacy of our method, showcasing a superior
performance in generating temporally consistent normal sequences with intricate
details from diverse videos.
67ff1cc4372d6790b1b7da90
Yanrui Bin
67ff1cc4372d6790b1b7da91
Wenbo Hu
claimed_verified
2025-04-16T11:57:59.932Z
657a7458afbb0117ba15c59f
https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
Wenbo Hu
user
wbhu-tc
67ff1cc4372d6790b1b7da92
Haoyuan Wang
67ff1cc4372d6790b1b7da93
Xinya Chen
67ff1cc4372d6790b1b7da94
Bing Wang
NormalCrafter: Learning Temporally Consistent Normals from Video
  Diffusion Priors
Surface normal estimation serves as a cornerstone for a spectrum of computer
vision applications. While numerous efforts have been devoted to static image
scenarios, ensuring temporal coherence in video-based normal estimation remains
a formidable challenge. Instead of merely augmenting existing methods with
temporal components, we present NormalCrafter to leverage the inherent temporal
priors of video diffusion models. To secure high-fidelity normal estimation
across sequences, we propose Semantic Feature Regularization (SFR), which
aligns diffusion features with semantic cues, encouraging the model to
concentrate on the intrinsic semantics of the scene. Moreover, we introduce a
two-stage training protocol that leverages both latent and pixel space learning
to preserve spatial accuracy while maintaining long temporal context. Extensive
evaluations demonstrate the efficacy of our method, showcasing a superior
performance in generating temporally consistent normal sequences with intricate
details from diverse videos.
A novel method, NormalCrafter, enhances temporal coherence in video-based surface normal estimation by utilizing video diffusion models with Semantic Feature Regularization and two-stage training in latent and pixel spaces.
video diffusion models Semantic Feature Regularization two-stage training latent space pixel space surface normal estimation temporal coherence