MIRAGE: Multimodal foundation model and benchmark for comprehensive
  retinal OCT image analysis
Artificial intelligence (AI) has become a fundamental tool for assisting
clinicians in analyzing ophthalmic images, such as optical coherence tomography
(OCT). However, developing AI models often requires extensive annotation, and
existing models tend to underperform on independent, unseen data. Foundation
models (FMs), large AI models trained on vast unlabeled datasets, have shown
promise in overcoming these challenges. Nonetheless, available FMs for
ophthalmology lack extensive validation, especially for segmentation tasks, and
focus on a single imaging modality. In this context, we propose MIRAGE, a novel
multimodal FM for the analysis of OCT and scanning laser ophthalmoscopy (SLO)
images. Additionally, we propose a new evaluation benchmark with OCT/SLO
classification and segmentation tasks. The comparison with general and
specialized FMs and segmentation methods shows the superiority of MIRAGE in
both types of tasks, highlighting its suitability as a basis for the
development of robust AI systems for retinal OCT image analysis. Both MIRAGE
and the evaluation benchmark are publicly available:
https://github.com/j-morano/MIRAGE.
684a96ab9aebf043cf7bdb2e
José Morano
claimed_verified
2025-06-12T12:40:45.632Z
655b3383ed8df831286969f0
/avatars/ab47ac437d9e978a7480cb9d82fa7638.svg
José Morano
user
j-morano
684a96ab9aebf043cf7bdb2f
Botond Fazekas
684a96ab9aebf043cf7bdb30
Emese Sükei
684a96ab9aebf043cf7bdb31
Ronald Fecso
684a96ab9aebf043cf7bdb32
Taha Emre
684a96ab9aebf043cf7bdb33
Markus Gumpinger
684a96ab9aebf043cf7bdb34
Georg Faustmann
684a96ab9aebf043cf7bdb35
Marzieh Oghbaie
684a96ab9aebf043cf7bdb36
Ursula Schmidt-Erfurth
684a96ab9aebf043cf7bdb37
Hrvoje Bogunović
MIRAGE: Multimodal foundation model and benchmark for comprehensive
  retinal OCT image analysis
Artificial intelligence (AI) has become a fundamental tool for assisting
clinicians in analyzing ophthalmic images, such as optical coherence tomography
(OCT). However, developing AI models often requires extensive annotation, and
existing models tend to underperform on independent, unseen data. Foundation
models (FMs), large AI models trained on vast unlabeled datasets, have shown
promise in overcoming these challenges. Nonetheless, available FMs for
ophthalmology lack extensive validation, especially for segmentation tasks, and
focus on a single imaging modality. In this context, we propose MIRAGE, a novel
multimodal FM for the analysis of OCT and scanning laser ophthalmoscopy (SLO)
images. Additionally, we propose a new evaluation benchmark with OCT/SLO
classification and segmentation tasks. The comparison with general and
specialized FMs and segmentation methods shows the superiority of MIRAGE in
both types of tasks, highlighting its suitability as a basis for the
development of robust AI systems for retinal OCT image analysis. Both MIRAGE
and the evaluation benchmark are publicly available:
https://github.com/j-morano/MIRAGE.
MIRAGE, a multimodal foundation model, excels in OCT and SLO image classification and segmentation, outperforming existing general and specialized models.
foundation models multimodal models OCT SLO segmentation tasks evaluation benchmark