Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object
  Structure via HyperNetworks
Solving image-to-3D from a single view is an ill-posed problem, and current
neural reconstruction methods addressing it through diffusion models still rely
on scene-specific optimization, constraining their generalization capability.
To overcome the limitations of existing approaches regarding generalization and
consistency, we introduce a novel neural rendering technique. Our approach
employs the signed distance function as the surface representation and
incorporates generalizable priors through geometry-encoding volumes and
HyperNetworks. Specifically, our method builds neural encoding volumes from
generated multi-view inputs. We adjust the weights of the SDF network
conditioned on an input image at test-time to allow model adaptation to novel
scenes in a feed-forward manner via HyperNetworks. To mitigate artifacts
derived from the synthesized views, we propose the use of a volume transformer
module to improve the aggregation of image features instead of processing each
viewpoint separately. Through our proposed method, dubbed as Hyper-VolTran, we
avoid the bottleneck of scene-specific optimization and maintain consistency
across the images generated from multiple viewpoints. Our experiments show the
advantages of our proposed approach with consistent results and rapid
generation.
658e58bdf0152a21fc99eaa1
Christian Simon
claimed_verified
2025-03-12T14:26:54.349Z
63394c4b20c058d8e235a133
/avatars/c6203765918ff588bfcf857d1d5adecc.svg
christian simon
user
cssen
658e58bdf0152a21fc99eaa2
Sen He
658e58bdf0152a21fc99eaa3
Juan-Manuel Perez-Rua
claimed_verified
2024-01-03T14:45:47.967Z
6572f4d86a1344d777fa7535
/avatars/eb19a0d4e8d8019456a78a7d621d0955.svg
Juanma Perez
user
jperezrua
658e58bdf0152a21fc99eaa4
Frost Xu
658e58bdf0152a21fc99eaa5
Amine Benhalloum
658e58bdf0152a21fc99eaa6
Tao Xiang
Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object
  Structure via HyperNetworks
Solving image-to-3D from a single view is an ill-posed problem, and current
neural reconstruction methods addressing it through diffusion models still rely
on scene-specific optimization, constraining their generalization capability.
To overcome the limitations of existing approaches regarding generalization and
consistency, we introduce a novel neural rendering technique. Our approach
employs the signed distance function as the surface representation and
incorporates generalizable priors through geometry-encoding volumes and
HyperNetworks. Specifically, our method builds neural encoding volumes from
generated multi-view inputs. We adjust the weights of the SDF network
conditioned on an input image at test-time to allow model adaptation to novel
scenes in a feed-forward manner via HyperNetworks. To mitigate artifacts
derived from the synthesized views, we propose the use of a volume transformer
module to improve the aggregation of image features instead of processing each
viewpoint separately. Through our proposed method, dubbed as Hyper-VolTran, we
avoid the bottleneck of scene-specific optimization and maintain consistency
across the images generated from multiple viewpoints. Our experiments show the
advantages of our proposed approach with consistent results and rapid
generation.
A neural rendering technique using signed distance functions, geometry-encoding volumes, and HyperNetworks achieves scene generalization and consistency without scene-specific optimization.
signed distance function geometry-encoding volumes HyperNetworks volume transformer Hyper-VolTran scene generalization consistency