Novel Object 6D Pose Estimation with a Single Reference View
Existing novel object 6D pose estimation methods typically rely on CAD models
or dense reference views, which are both difficult to acquire. Using only a
single reference view is more scalable, but challenging due to large pose
discrepancies and limited geometric and spatial information. To address these
issues, we propose a Single-Reference-based novel object 6D (SinRef-6D) pose
estimation method. Our key idea is to iteratively establish point-wise
alignment in the camera coordinate system based on state space models (SSMs).
Specifically, iterative camera-space point-wise alignment can effectively
handle large pose discrepancies, while our proposed RGB and Points SSMs can
capture long-range dependencies and spatial information from a single view,
offering linear complexity and superior spatial modeling capability. Once
pre-trained on synthetic data, SinRef-6D can estimate the 6D pose of a novel
object using only a single reference view, without requiring retraining or a
CAD model. Extensive experiments on six popular datasets and real-world robotic
scenes demonstrate that we achieve on-par performance with CAD-based and dense
reference view-based methods, despite operating in the more challenging single
reference setting. Code will be released at
https://github.com/CNJianLiu/SinRef-6D.
67d01e7b66fafa3aa91677e1
Jian Liu
claimed_verified
2025-03-11T11:43:04.337Z
67d01cc5211e07ebcf89efd0
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/P2Czu4w0Zz-POjteCtEm_.jpeg
Jian Liu
user
JianLiu99
67d01e7b66fafa3aa91677e2
Wei Sun
67d01e7b66fafa3aa91677e3
Kai Zeng
67d01e7b66fafa3aa91677e4
Jin Zheng
67d01e7b66fafa3aa91677e5
Hui Yang
67d01e7b66fafa3aa91677e6
Lin Wang
67d01e7b66fafa3aa91677e7
Hossein Rahmani
67d01e7b66fafa3aa91677e8
Ajmal Mian
Novel Object 6D Pose Estimation with a Single Reference View
Existing novel object 6D pose estimation methods typically rely on CAD models
or dense reference views, which are both difficult to acquire. Using only a
single reference view is more scalable, but challenging due to large pose
discrepancies and limited geometric and spatial information. To address these
issues, we propose a Single-Reference-based novel object 6D (SinRef-6D) pose
estimation method. Our key idea is to iteratively establish point-wise
alignment in the camera coordinate system based on state space models (SSMs).
Specifically, iterative camera-space point-wise alignment can effectively
handle large pose discrepancies, while our proposed RGB and Points SSMs can
capture long-range dependencies and spatial information from a single view,
offering linear complexity and superior spatial modeling capability. Once
pre-trained on synthetic data, SinRef-6D can estimate the 6D pose of a novel
object using only a single reference view, without requiring retraining or a
CAD model. Extensive experiments on six popular datasets and real-world robotic
scenes demonstrate that we achieve on-par performance with CAD-based and dense
reference view-based methods, despite operating in the more challenging single
reference setting. Code will be released at
https://github.com/CNJianLiu/SinRef-6D.
SinRef-6D is a single-reference-based method for 6D pose estimation that uses state space models to align points and capture long-range dependencies, achieving performance comparable to methods using multiple references or CAD models.
state space models SSMs point-wise alignment camera coordinate system long-range dependencies 6D pose estimation SinRef-6D