SAMed-2: Selective Memory Enhanced Medical Segment Anything Model
Recent "segment anything" efforts show promise by learning from large-scale
data, but adapting such models directly to medical images remains challenging
due to the complexity of medical data, noisy annotations, and continual
learning requirements across diverse modalities and anatomical structures. In
this work, we propose SAMed-2, a new foundation model for medical image
segmentation built upon the SAM-2 architecture. Specifically, we introduce a
temporal adapter into the image encoder to capture image correlations and a
confidence-driven memory mechanism to store high-certainty features for later
retrieval. This memory-based strategy counters the pervasive noise in
large-scale medical datasets and mitigates catastrophic forgetting when
encountering new tasks or modalities. To train and evaluate SAMed-2, we curate
MedBank-100k, a comprehensive dataset spanning seven imaging modalities and 21
medical segmentation tasks. Our experiments on both internal benchmarks and 10
external datasets demonstrate superior performance over state-of-the-art
baselines in multi-task scenarios. The code is available at:
https://github.com/ZhilingYan/Medical-SAM-Bench.
686dc7ebcb5725779c60b35d
Zhiling Yan
686dc7ebcb5725779c60b35e
Sifan Song
686dc7ebcb5725779c60b35f
Dingjie Song
claimed_verified
2025-07-09T08:50:29.991Z
619f01b8cc04eadf54fa5d5d
/avatars/928f3d1a6146e2e1ae4860445d929d5c.svg
Song Dingjie
user
songdj
686dc7ebcb5725779c60b360
Yiwei Li
686dc7ebcb5725779c60b361
Rong Zhou
686dc7ebcb5725779c60b362
Weixiang Sun
claimed_verified
2025-07-09T14:41:02.920Z
6481a16f70ac5e1968a7bb97
https://cdn-avatars.huggingface.co/v1/production/uploads/6481a16f70ac5e1968a7bb97/ith2d4CuhfJH1CeU92wzE.jpeg
Weixiang Sun
user
Sweson
686dc7ebcb5725779c60b363
Zhennong Chen
686dc7ebcb5725779c60b364
Sekeun Kim
686dc7ebcb5725779c60b365
Hui Ren
686dc7ebcb5725779c60b366
Tianming Liu
686dc7ebcb5725779c60b367
Quanzheng Li
686dc7ebcb5725779c60b368
Xiang Li
686dc7ebcb5725779c60b369
Lifang He
686dc7ebcb5725779c60b36a
Lichao Sun
SAMed-2: Selective Memory Enhanced Medical Segment Anything Model
Recent "segment anything" efforts show promise by learning from large-scale
data, but adapting such models directly to medical images remains challenging
due to the complexity of medical data, noisy annotations, and continual
learning requirements across diverse modalities and anatomical structures. In
this work, we propose SAMed-2, a new foundation model for medical image
segmentation built upon the SAM-2 architecture. Specifically, we introduce a
temporal adapter into the image encoder to capture image correlations and a
confidence-driven memory mechanism to store high-certainty features for later
retrieval. This memory-based strategy counters the pervasive noise in
large-scale medical datasets and mitigates catastrophic forgetting when
encountering new tasks or modalities. To train and evaluate SAMed-2, we curate
MedBank-100k, a comprehensive dataset spanning seven imaging modalities and 21
medical segmentation tasks. Our experiments on both internal benchmarks and 10
external datasets demonstrate superior performance over state-of-the-art
baselines in multi-task scenarios. The code is available at:
https://github.com/ZhilingYan/Medical-SAM-Bench.
SAMed-2, an adaptation of SAM-2 for medical image segmentation, incorporates a temporal adapter and confidence-driven memory to handle noise and catastrophic forgetting, achieving superior performance across multiple tasks and modalities.
SAM-2 temporal adapter confidence-driven memory catastrophic forgetting MedBank-100k medical segmentation tasks