Towards More Diverse and Challenging Pre-training for Point Cloud
  Learning: Self-Supervised Cross Reconstruction with Decoupled Views
Point cloud learning, especially in a self-supervised way without manual
labels, has gained growing attention in both vision and learning communities
due to its potential utility in a wide range of applications. Most existing
generative approaches for point cloud self-supervised learning focus on
recovering masked points from visible ones within a single view. Recognizing
that a two-view pre-training paradigm inherently introduces greater diversity
and variance, it may thus enable more challenging and informative pre-training.
Inspired by this, we explore the potential of two-view learning in this domain.
In this paper, we propose Point-PQAE, a cross-reconstruction generative
paradigm that first generates two decoupled point clouds/views and then
reconstructs one from the other. To achieve this goal, we develop a crop
mechanism for point cloud view generation for the first time and further
propose a novel positional encoding to represent the 3D relative position
between the two decoupled views. The cross-reconstruction significantly
increases the difficulty of pre-training compared to self-reconstruction, which
enables our method to surpass previous single-modal self-reconstruction methods
in 3D self-supervised learning. Specifically, it outperforms the
self-reconstruction baseline (Point-MAE) by 6.5%, 7.0%, and 6.7% in three
variants of ScanObjectNN with the Mlp-Linear evaluation protocol. The code is
available at https://github.com/aHapBean/Point-PQAE.
68b7ba97295f15ff6091133f
Xiangdong Zhang
68b7ba97295f15ff60911340
Shaofeng Zhang
68b7ba97295f15ff60911341
Junchi Yan
Towards More Diverse and Challenging Pre-training for Point Cloud
  Learning: Self-Supervised Cross Reconstruction with Decoupled Views
Point cloud learning, especially in a self-supervised way without manual
labels, has gained growing attention in both vision and learning communities
due to its potential utility in a wide range of applications. Most existing
generative approaches for point cloud self-supervised learning focus on
recovering masked points from visible ones within a single view. Recognizing
that a two-view pre-training paradigm inherently introduces greater diversity
and variance, it may thus enable more challenging and informative pre-training.
Inspired by this, we explore the potential of two-view learning in this domain.
In this paper, we propose Point-PQAE, a cross-reconstruction generative
paradigm that first generates two decoupled point clouds/views and then
reconstructs one from the other. To achieve this goal, we develop a crop
mechanism for point cloud view generation for the first time and further
propose a novel positional encoding to represent the 3D relative position
between the two decoupled views. The cross-reconstruction significantly
increases the difficulty of pre-training compared to self-reconstruction, which
enables our method to surpass previous single-modal self-reconstruction methods
in 3D self-supervised learning. Specifically, it outperforms the
self-reconstruction baseline (Point-MAE) by 6.5%, 7.0%, and 6.7% in three
variants of ScanObjectNN with the Mlp-Linear evaluation protocol. The code is
available at https://github.com/aHapBean/Point-PQAE.
Point-PQAE, a two-view cross-reconstruction generative paradigm, enhances 3D self-supervised learning by introducing greater diversity and variance, outperforming single-view methods in point cloud reconstruction tasks.
point cloud learning self-supervised learning generative approaches two-view pre-training cross-reconstruction crop mechanism positional encoding 3D relative position ScanObjectNN Mlp-Linear evaluation protocol