RGB-Only Supervised Camera Parameter Optimization in Dynamic Scenes
Although COLMAP has long remained the predominant method for camera parameter
optimization in static scenes, it is constrained by its lengthy runtime and
reliance on ground truth (GT) motion masks for application to dynamic scenes.
Many efforts attempted to improve it by incorporating more priors as
supervision such as GT focal length, motion masks, 3D point clouds, camera
poses, and metric depth, which, however, are typically unavailable in casually
captured RGB videos. In this paper, we propose a novel method for more accurate
and efficient camera parameter optimization in dynamic scenes solely supervised
by a single RGB video. Our method consists of three key components: (1)
Patch-wise Tracking Filters, to establish robust and maximally sparse
hinge-like relations across the RGB video. (2) Outlier-aware Joint
Optimization, for efficient camera parameter optimization by adaptive
down-weighting of moving outliers, without reliance on motion priors. (3) A
Two-stage Optimization Strategy, to enhance stability and optimization speed by
a trade-off between the Softplus limits and convex minima in losses. We
visually and numerically evaluate our camera estimates. To further validate
accuracy, we feed the camera estimates into a 4D reconstruction method and
assess the resulting 3D scenes, and rendered 2D RGB and depth maps. We perform
experiments on 4 real-world datasets (NeRF-DS, DAVIS, iPhone, and TUM-dynamics)
and 1 synthetic dataset (MPI-Sintel), demonstrating that our method estimates
camera parameters more efficiently and accurately with a single RGB video as
the only supervision.
68ce18078adc5cd018d15782
Fang Li
claimed_verified
2025-09-21T13:12:26.682Z
6854d82d08d8c86c86fc5f03
/avatars/cf6a6ad74053bc4668766c132a18df2d.svg
Fang Li
user
fangli3
68ce18078adc5cd018d15783
Hao Zhang
68ce18078adc5cd018d15784
Narendra Ahuja
RGB-Only Supervised Camera Parameter Optimization in Dynamic Scenes
Although COLMAP has long remained the predominant method for camera parameter
optimization in static scenes, it is constrained by its lengthy runtime and
reliance on ground truth (GT) motion masks for application to dynamic scenes.
Many efforts attempted to improve it by incorporating more priors as
supervision such as GT focal length, motion masks, 3D point clouds, camera
poses, and metric depth, which, however, are typically unavailable in casually
captured RGB videos. In this paper, we propose a novel method for more accurate
and efficient camera parameter optimization in dynamic scenes solely supervised
by a single RGB video. Our method consists of three key components: (1)
Patch-wise Tracking Filters, to establish robust and maximally sparse
hinge-like relations across the RGB video. (2) Outlier-aware Joint
Optimization, for efficient camera parameter optimization by adaptive
down-weighting of moving outliers, without reliance on motion priors. (3) A
Two-stage Optimization Strategy, to enhance stability and optimization speed by
a trade-off between the Softplus limits and convex minima in losses. We
visually and numerically evaluate our camera estimates. To further validate
accuracy, we feed the camera estimates into a 4D reconstruction method and
assess the resulting 3D scenes, and rendered 2D RGB and depth maps. We perform
experiments on 4 real-world datasets (NeRF-DS, DAVIS, iPhone, and TUM-dynamics)
and 1 synthetic dataset (MPI-Sintel), demonstrating that our method estimates
camera parameters more efficiently and accurately with a single RGB video as
the only supervision.
A novel method for camera parameter optimization in dynamic scenes using a single RGB video, incorporating patch-wise tracking filters, outlier-aware joint optimization, and a two-stage optimization strategy.
Patch-wise Tracking Filters Outlier-aware Joint Optimization Two-stage Optimization Strategy Softplus limits convex minima 4D reconstruction NeRF-DS DAVIS iPhone TUM-dynamics MPI-Sintel