R-MAE: Regions Meet Masked Autoencoders
Vision-specific concepts such as "region" have played a key role in extending
general machine learning frameworks to tasks like object detection. Given the
success of region-based detectors for supervised learning and the progress of
intra-image methods for contrastive learning, we explore the use of regions for
reconstructive pre-training. Starting from Masked Autoencoding (MAE) both as a
baseline and an inspiration, we propose a parallel pre-text task tailored to
address the one-to-many mapping between images and regions. Since such regions
can be generated in an unsupervised way, our approach (R-MAE) inherits the wide
applicability from MAE, while being more "region-aware". We conduct thorough
analyses during the development of R-MAE, and converge on a variant that is
both effective and efficient (1.3% overhead over MAE). Moreover, it shows
consistent quantitative improvements when generalized to various pre-training
data and downstream detection and segmentation benchmarks. Finally, we provide
extensive qualitative visualizations to enhance the understanding of R-MAE's
behaviour and potential. Code will be made available at
https://github.com/facebookresearch/r-mae.
6482c107474d4d463d4eee53
Duy-Kien Nguyen
admin_assigned
2023-06-12T13:09:02.154Z
6317dfa31dd5a319a67f2556
/avatars/4a6b5748b8d95a15be560af5fd2de998.svg
Kien Nguyen
user
duykien
6482c107474d4d463d4eee54
Vaibhav Aggarwal
admin_assigned
2023-06-12T13:10:14.815Z
629f97b73fa7e9d614e92212
/avatars/6903732275dc71213cfbf82a98c39084.svg
Vaibhav Aggarwal
user
vaibhavagg303
6482c107474d4d463d4eee55
Yanghao Li
6482c107474d4d463d4eee56
Martin R. Oswald
6482c107474d4d463d4eee57
Alexander Kirillov
admin_assigned
2023-06-12T13:11:21.278Z
630fa75802ce39336c45f31c
/avatars/0e4e3839a4e6f182c2618eb35acef29a.svg
Alexander Kirillov
user
akirillov
6482c107474d4d463d4eee58
Cees G. M. Snoek
6482c107474d4d463d4eee59
Xinlei Chen
claimed_verified
2023-06-12T09:11:53.653Z
63e58e3a006a775275e59e41
/avatars/75262a35b27a2ae1939df9118120d99e.svg
Xinlei Chen
user
endernewton
R-MAE: Regions Meet Masked Autoencoders
Vision-specific concepts such as "region" have played a key role in extending
general machine learning frameworks to tasks like object detection. Given the
success of region-based detectors for supervised learning and the progress of
intra-image methods for contrastive learning, we explore the use of regions for
reconstructive pre-training. Starting from Masked Autoencoding (MAE) both as a
baseline and an inspiration, we propose a parallel pre-text task tailored to
address the one-to-many mapping between images and regions. Since such regions
can be generated in an unsupervised way, our approach (R-MAE) inherits the wide
applicability from MAE, while being more "region-aware". We conduct thorough
analyses during the development of R-MAE, and converge on a variant that is
both effective and efficient (1.3% overhead over MAE). Moreover, it shows
consistent quantitative improvements when generalized to various pre-training
data and downstream detection and segmentation benchmarks. Finally, we provide
extensive qualitative visualizations to enhance the understanding of R-MAE's
behaviour and potential. Code will be made available at
https://github.com/facebookresearch/r-mae.
R-MAE, an extension of Masked Autoencoding tailored for region-based pre-training, improves unsupervised learning in computer vision tasks with minimal overhead.
Masked Autoencoding region-based pre-training R-MAE object detection contrastive learning intra-image methods one-to-many mapping region-aware pre-training data downstream detection segmentation benchmarks