Neural Scene Chronology
In this work, we aim to reconstruct a time-varying 3D model, capable of
rendering photo-realistic renderings with independent control of viewpoint,
illumination, and time, from Internet photos of large-scale landmarks. The core
challenges are twofold. First, different types of temporal changes, such as
illumination and changes to the underlying scene itself (such as replacing one
graffiti artwork with another) are entangled together in the imagery. Second,
scene-level temporal changes are often discrete and sporadic over time, rather
than continuous. To tackle these problems, we propose a new scene
representation equipped with a novel temporal step function encoding method
that can model discrete scene-level content changes as piece-wise constant
functions over time. Specifically, we represent the scene as a space-time
radiance field with a per-image illumination embedding, where
temporally-varying scene changes are encoded using a set of learned step
functions. To facilitate our task of chronology reconstruction from Internet
imagery, we also collect a new dataset of four scenes that exhibit various
changes over time. We demonstrate that our method exhibits state-of-the-art
view synthesis results on this dataset, while achieving independent control of
viewpoint, time, and illumination.
64893e904b47b34bd3d41cac
Haotong Lin
claimed_verified
2023-06-14T12:15:16.943Z
6489a01b8de3f9d810b0154f
/avatars/f7a0fc6816535945e11bac1212dd7b57.svg
Haotong Lin
user
haotongl
64893e904b47b34bd3d41cad
Qianqian Wang
64893e904b47b34bd3d41cae
Ruojin Cai
64893e904b47b34bd3d41caf
Sida Peng
admin_assigned
2023-06-14T09:20:00.215Z
62986ca2b58e71e2ac9b8f01
/avatars/83944db5f3dbb6f47c47c46fb2cb2849.svg
Sida Peng
user
pengsida
64893e904b47b34bd3d41cb0
Hadar Averbuch-Elor
64893e904b47b34bd3d41cb1
Xiaowei Zhou
64893e904b47b34bd3d41cb2
Noah Snavely
admin_assigned
2023-06-14T09:19:47.227Z
632b79ca0a9dc02985af9e0b
/avatars/e2ca585398a3212b2956a14804d0ba67.svg
Noah Snavely
user
jimantha
Neural Scene Chronology
In this work, we aim to reconstruct a time-varying 3D model, capable of
rendering photo-realistic renderings with independent control of viewpoint,
illumination, and time, from Internet photos of large-scale landmarks. The core
challenges are twofold. First, different types of temporal changes, such as
illumination and changes to the underlying scene itself (such as replacing one
graffiti artwork with another) are entangled together in the imagery. Second,
scene-level temporal changes are often discrete and sporadic over time, rather
than continuous. To tackle these problems, we propose a new scene
representation equipped with a novel temporal step function encoding method
that can model discrete scene-level content changes as piece-wise constant
functions over time. Specifically, we represent the scene as a space-time
radiance field with a per-image illumination embedding, where
temporally-varying scene changes are encoded using a set of learned step
functions. To facilitate our task of chronology reconstruction from Internet
imagery, we also collect a new dataset of four scenes that exhibit various
changes over time. We demonstrate that our method exhibits state-of-the-art
view synthesis results on this dataset, while achieving independent control of
viewpoint, time, and illumination.
A novel scene representation with temporal step function encoding enables reconstruction of time-varying 3D models from internet photos, achieving state-of-the-art view synthesis with independent control over viewpoint, time, and illumination.
space-time radiance field illumination embedding step functions time-varying scene changes discrete content changes chronology reconstruction view synthesis