MedDINOv3: How to adapt vision foundation models for medical image
  segmentation?
Accurate segmentation of organs and tumors in CT and MRI scans is essential
for diagnosis, treatment planning, and disease monitoring. While deep learning
has advanced automated segmentation, most models remain task-specific, lacking
generalizability across modalities and institutions. Vision foundation models
(FMs) pretrained on billion-scale natural images offer powerful and
transferable representations. However, adapting them to medical imaging faces
two key challenges: (1) the ViT backbone of most foundation models still
underperform specialized CNNs on medical image segmentation, and (2) the large
domain gap between natural and medical images limits transferability. We
introduce MedDINOv3, a simple and effective framework for adapting
DINOv3 to medical segmentation. We first revisit plain ViTs and design a simple
and effective architecture with multi-scale token aggregation. Then, we perform
domain-adaptive pretraining on CT-3M, a curated collection of 3.87M
axial CT slices, using a multi-stage DINOv3 recipe to learn robust dense
features. MedDINOv3 matches or exceeds state-of-the-art performance across four
segmentation benchmarks, demonstrating the potential of vision foundation
models as unified backbones for medical image segmentation. The code is
available at https://github.com/ricklisz/MedDINOv3.
68b7bad9295f15ff60911344
Yuheng Li
claimed_verified
2025-09-03T19:25:37.793Z
684e06c0fa18169623729808
/avatars/91cdabc11429f68274293c7251aaf57f.svg
Yuheng Li
user
ricklisz123
68b7bad9295f15ff60911345
Yizhou Wu
admin_assigned
2025-09-03T09:04:52.458Z
66e3a2c836a20907db1ff956
/avatars/5c9c6203df62820278ab27751f6cb774.svg
Yizhou Wu
user
mcl0222
68b7bad9295f15ff60911346
Yuxiang Lai
admin_assigned
2025-09-03T09:04:58.733Z
671a569a97cb6656e4859ea8
/avatars/9eb7966b3ff39ee342e9e681256c86a7.svg
Yuxiang Lai
user
yuxianglai117
68b7bad9295f15ff60911347
Mingzhe Hu
admin_assigned
2025-09-03T09:05:03.944Z
68706a9d83cbc7f753c90841
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ysBWLAoRua5lY4qmVR5_L.png
Mingzhe Hu
user
wy20030128
68b7bad9295f15ff60911348
Xiaofeng Yang
MedDINOv3: How to adapt vision foundation models for medical image
  segmentation?
Accurate segmentation of organs and tumors in CT and MRI scans is essential
for diagnosis, treatment planning, and disease monitoring. While deep learning
has advanced automated segmentation, most models remain task-specific, lacking
generalizability across modalities and institutions. Vision foundation models
(FMs) pretrained on billion-scale natural images offer powerful and
transferable representations. However, adapting them to medical imaging faces
two key challenges: (1) the ViT backbone of most foundation models still
underperform specialized CNNs on medical image segmentation, and (2) the large
domain gap between natural and medical images limits transferability. We
introduce MedDINOv3, a simple and effective framework for adapting
DINOv3 to medical segmentation. We first revisit plain ViTs and design a simple
and effective architecture with multi-scale token aggregation. Then, we perform
domain-adaptive pretraining on CT-3M, a curated collection of 3.87M
axial CT slices, using a multi-stage DINOv3 recipe to learn robust dense
features. MedDINOv3 matches or exceeds state-of-the-art performance across four
segmentation benchmarks, demonstrating the potential of vision foundation
models as unified backbones for medical image segmentation. The code is
available at https://github.com/ricklisz/MedDINOv3.
MedDINOv3, a framework adapting DINOv3 with multi-scale token aggregation, achieves state-of-the-art performance in medical image segmentation by overcoming challenges in domain adaptation and backbone performance.
ViT CNN vision foundation models DINOv3 multi-scale token aggregation domain-adaptive pretraining CT-3M dense features medical image segmentation