Adapting Vision Foundation Models for Robust Cloud Segmentation in
  Remote Sensing Images
Cloud segmentation is a critical challenge in remote sensing image
interpretation, as its accuracy directly impacts the effectiveness of
subsequent data processing and analysis. Recently, vision foundation models
(VFM) have demonstrated powerful generalization capabilities across various
visual tasks. In this paper, we present a parameter-efficient adaptive
approach, termed Cloud-Adapter, designed to enhance the accuracy and robustness
of cloud segmentation. Our method leverages a VFM pretrained on general domain
data, which remains frozen, eliminating the need for additional training.
Cloud-Adapter incorporates a lightweight spatial perception module that
initially utilizes a convolutional neural network (ConvNet) to extract dense
spatial representations. These multi-scale features are then aggregated and
serve as contextual inputs to an adapting module, which modulates the frozen
transformer layers within the VFM. Experimental results demonstrate that the
Cloud-Adapter approach, utilizing only 0.6% of the trainable parameters of the
frozen backbone, achieves substantial performance gains. Cloud-Adapter
consistently attains state-of-the-art (SOTA) performance across a wide variety
of cloud segmentation datasets from multiple satellite sources, sensor series,
data processing levels, land cover scenarios, and annotation granularities. We
have released the source code and pretrained models at
https://github.com/XavierJiezou/Cloud-Adapter to support further research.
67416de111bf5a76d6eb47ca
Xuechao Zou
extracted_pending
2024-11-23T05:53:38.924Z
6617af2beab5eef6b1e8bb9e
/avatars/d939c02027916331d4c44119565f2ca6.svg
XavierJiezou
user
XavierJiezou
67416de111bf5a76d6eb47cb
Shun Zhang
67416de111bf5a76d6eb47cc
Kai Li
extracted_pending
2024-11-23T05:53:38.924Z
6387676c23da90491eb9fb16
https://cdn-avatars.huggingface.co/v1/production/uploads/1669818175965-noauth.jpeg
Kai Li
user
JusperLee
67416de111bf5a76d6eb47cd
Shiying Wang
admin_assigned
2024-11-25T20:58:29.826Z
64b519944e5902e2fdfbde6b
/avatars/e40f2cbd6ff1deec253016ddf76b2117.svg
wangshiying
user
shiyingstone
67416de111bf5a76d6eb47ce
Junliang Xing
67416de111bf5a76d6eb47cf
Lei Jin
67416de111bf5a76d6eb47d0
Congyan Lang
67416de111bf5a76d6eb47d1
Pin Tao
Adapting Vision Foundation Models for Robust Cloud Segmentation in
  Remote Sensing Images
Cloud segmentation is a critical challenge in remote sensing image
interpretation, as its accuracy directly impacts the effectiveness of
subsequent data processing and analysis. Recently, vision foundation models
(VFM) have demonstrated powerful generalization capabilities across various
visual tasks. In this paper, we present a parameter-efficient adaptive
approach, termed Cloud-Adapter, designed to enhance the accuracy and robustness
of cloud segmentation. Our method leverages a VFM pretrained on general domain
data, which remains frozen, eliminating the need for additional training.
Cloud-Adapter incorporates a lightweight spatial perception module that
initially utilizes a convolutional neural network (ConvNet) to extract dense
spatial representations. These multi-scale features are then aggregated and
serve as contextual inputs to an adapting module, which modulates the frozen
transformer layers within the VFM. Experimental results demonstrate that the
Cloud-Adapter approach, utilizing only 0.6% of the trainable parameters of the
frozen backbone, achieves substantial performance gains. Cloud-Adapter
consistently attains state-of-the-art (SOTA) performance across a wide variety
of cloud segmentation datasets from multiple satellite sources, sensor series,
data processing levels, land cover scenarios, and annotation granularities. We
have released the source code and pretrained models at
https://github.com/XavierJiezou/Cloud-Adapter to support further research.
The Cloud-Adapter approach enhances cloud segmentation accuracy using a parameter-efficient method that adapts a pretrained vision foundation model with a lightweight spatial perception module.
vision foundation models (VFM) parameter-efficient Cloud-Adapter convolutional neural network (ConvNet) spatial perception module adapting module frozen transformer layers state-of-the-art (SOTA)