EscherNet: A Generative Model for Scalable View Synthesis
We introduce EscherNet, a multi-view conditioned diffusion model for view
synthesis. EscherNet learns implicit and generative 3D representations coupled
with a specialised camera positional encoding, allowing precise and continuous
relative control of the camera transformation between an arbitrary number of
reference and target views. EscherNet offers exceptional generality,
flexibility, and scalability in view synthesis -- it can generate more than 100
consistent target views simultaneously on a single consumer-grade GPU, despite
being trained with a fixed number of 3 reference views to 3 target views. As a
result, EscherNet not only addresses zero-shot novel view synthesis, but also
naturally unifies single- and multi-image 3D reconstruction, combining these
diverse tasks into a single, cohesive framework. Our extensive experiments
demonstrate that EscherNet achieves state-of-the-art performance in multiple
benchmarks, even when compared to methods specifically tailored for each
individual problem. This remarkable versatility opens up new directions for
designing scalable neural architectures for 3D vision. Project page:
https://kxhit.github.io/EscherNet.
65c306fe400d860a3eae7e9a
Xin Kong
claimed_verified
2024-02-07T20:41:00.936Z
633ef0bfac395449287b3ed9
/avatars/7e2cc27da7761f4e7af547b7d65528db.svg
Xin Kong
user
kxic
65c306fe400d860a3eae7e9b
Shikun Liu
claimed_verified
2024-02-08T11:52:52.000Z
633ef2d6905131c8f52cfb95
/avatars/415f61f80feaffca4e35e029b84ba5d2.svg
Shikun Liu
user
shikunl
65c306fe400d860a3eae7e9c
Xiaoyang Lyu
65c306fe400d860a3eae7e9d
Marwan Taher
claimed_verified
2024-06-06T06:56:23.960Z
633732015f0025d728b830fe
/avatars/59a38b26604e08bc3e6509600d002bfe.svg
Marwan Taher
user
marwanPtr
65c306fe400d860a3eae7e9e
Xiaojuan Qi
claimed_verified
2025-07-15T19:15:44.611Z
6875266f9cd3191dfddc7071
/avatars/64c581910833b111e9a7bae5b8740229.svg
xiaojuan qi
user
xjqi
65c306fe400d860a3eae7e9f
Andrew J. Davison
EscherNet: A Generative Model for Scalable View Synthesis
We introduce EscherNet, a multi-view conditioned diffusion model for view
synthesis. EscherNet learns implicit and generative 3D representations coupled
with a specialised camera positional encoding, allowing precise and continuous
relative control of the camera transformation between an arbitrary number of
reference and target views. EscherNet offers exceptional generality,
flexibility, and scalability in view synthesis -- it can generate more than 100
consistent target views simultaneously on a single consumer-grade GPU, despite
being trained with a fixed number of 3 reference views to 3 target views. As a
result, EscherNet not only addresses zero-shot novel view synthesis, but also
naturally unifies single- and multi-image 3D reconstruction, combining these
diverse tasks into a single, cohesive framework. Our extensive experiments
demonstrate that EscherNet achieves state-of-the-art performance in multiple
benchmarks, even when compared to methods specifically tailored for each
individual problem. This remarkable versatility opens up new directions for
designing scalable neural architectures for 3D vision. Project page:
https://kxhit.github.io/EscherNet.
EscherNet, a multi-view conditioned diffusion model with specialized camera encoding, performs exceptional view synthesis, achieving state-of-the-art in benchmarks and combining synthesis and reconstruction tasks.
diffusion model view synthesis implicit 3D representations generative 3D representations camera positional encoding camera transformation zero-shot novel view synthesis single-image 3D reconstruction multi-image 3D reconstruction