ICON: Incremental CONfidence for Joint Pose and Radiance Field
  Optimization
Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View
Synthesis (NVS) given a set of 2D images. However, NeRF training requires
accurate camera pose for each input view, typically obtained by
Structure-from-Motion (SfM) pipelines. Recent works have attempted to relax
this constraint, but they still often rely on decent initial poses which they
can refine. Here we aim at removing the requirement for pose initialization. We
present Incremental CONfidence (ICON), an optimization procedure for training
NeRFs from 2D video frames. ICON only assumes smooth camera motion to estimate
initial guess for poses. Further, ICON introduces ``confidence": an adaptive
measure of model quality used to dynamically reweight gradients. ICON relies on
high-confidence poses to learn NeRF, and high-confidence 3D structure (as
encoded by NeRF) to learn poses. We show that ICON, without prior pose
initialization, achieves superior performance in both CO3D and HO3D versus
methods which use SfM pose.
65a8ab47a6fe31817b19f30e
Weiyao Wang
extracted_confirmed
2024-01-23T02:09:15.983Z
65af1f30de38fbe9229f1860
/avatars/30c3bcdb441cf3aff475f1769900df58.svg
Weiyao Wang
user
weiyaow
65a8ab47a6fe31817b19f30f
Pierre Gleize
claimed_verified
2024-03-14T08:00:18.930Z
65f22480f5cf26fe060cb558
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/2XwjrbcFiBJoO9hqcsniI.jpeg
Pierre Gleize
user
pgleize
65a8ab47a6fe31817b19f310
Hao Tang
65a8ab47a6fe31817b19f311
Xingyu Chen
65a8ab47a6fe31817b19f312
Kevin J Liang
claimed_verified
2024-03-14T16:31:52.795Z
65f3227730d1859ef2ab9c3f
/avatars/fea5047893056a8bc73882b7fa1d9b5e.svg
Kevin J Liang
user
kevinjliang
65a8ab47a6fe31817b19f313
Matt Feiszli
claimed_verified
2024-01-23T08:09:39.484Z
65a91cda241e1c6c489d53bf
/avatars/eb864f738db56b7e00f40a66d2e3af6c.svg
Matt Feiszli
user
mattfeiszli
ICON: Incremental CONfidence for Joint Pose and Radiance Field
  Optimization
Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View
Synthesis (NVS) given a set of 2D images. However, NeRF training requires
accurate camera pose for each input view, typically obtained by
Structure-from-Motion (SfM) pipelines. Recent works have attempted to relax
this constraint, but they still often rely on decent initial poses which they
can refine. Here we aim at removing the requirement for pose initialization. We
present Incremental CONfidence (ICON), an optimization procedure for training
NeRFs from 2D video frames. ICON only assumes smooth camera motion to estimate
initial guess for poses. Further, ICON introduces ``confidence": an adaptive
measure of model quality used to dynamically reweight gradients. ICON relies on
high-confidence poses to learn NeRF, and high-confidence 3D structure (as
encoded by NeRF) to learn poses. We show that ICON, without prior pose
initialization, achieves superior performance in both CO3D and HO3D versus
methods which use SfM pose.
Incremental CONfidence (ICON) allows NeRF training from 2D video frames without initial pose estimation, utilizing adaptive confidence weighting to achieve superior performance in novel view synthesis.
Neural Radiance Fields (NeRF) Novel View Synthesis (NVS) Structure-from-Motion (SfM) ICON optimization procedure camera motion confidence adaptive measure model quality gradient weighting high-confidence poses high-confidence 3D structure CO3D HO3D