EchoPrime: A Multi-Video View-Informed Vision-Language Model for
  Comprehensive Echocardiography Interpretation
Echocardiography is the most widely used cardiac imaging modality, capturing
ultrasound video data to assess cardiac structure and function. Artificial
intelligence (AI) in echocardiography has the potential to streamline manual
tasks and improve reproducibility and precision. However, most echocardiography
AI models are single-view, single-task systems that do not synthesize
complementary information from multiple views captured during a full exam, and
thus lead to limited performance and scope of applications. To address this
problem, we introduce EchoPrime, a multi-view, view-informed, video-based
vision-language foundation model trained on over 12 million video-report pairs.
EchoPrime uses contrastive learning to train a unified embedding model for all
standard views in a comprehensive echocardiogram study with representation of
both rare and common diseases and diagnoses. EchoPrime then utilizes
view-classification and a view-informed anatomic attention model to weight
video-specific interpretations that accurately maps the relationship between
echocardiographic views and anatomical structures. With retrieval-augmented
interpretation, EchoPrime integrates information from all echocardiogram videos
in a comprehensive study and performs holistic comprehensive clinical
echocardiography interpretation. In datasets from two independent healthcare
systems, EchoPrime achieves state-of-the art performance on 23 diverse
benchmarks of cardiac form and function, surpassing the performance of both
task-specific approaches and prior foundation models. Following rigorous
clinical evaluation, EchoPrime can assist physicians in the automated
preliminary assessment of comprehensive echocardiography.
670ee7064d88c510f47ca128
Milos Vukadinovic
claimed_verified
2024-10-16T09:44:12.113Z
64fb4eca0e486522f81fba18
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/OKC0X55IYBsDfVJhDyc5v.png
Milos Vukadinovic
user
milosvuk
670ee7064d88c510f47ca129
Xiu Tang
670ee7064d88c510f47ca12a
Neal Yuan
670ee7064d88c510f47ca12b
Paul Cheng
670ee7064d88c510f47ca12c
Debiao Li
670ee7064d88c510f47ca12d
Susan Cheng
670ee7064d88c510f47ca12e
Bryan He
670ee7064d88c510f47ca12f
David Ouyang
EchoPrime: A Multi-Video View-Informed Vision-Language Model for
  Comprehensive Echocardiography Interpretation
Echocardiography is the most widely used cardiac imaging modality, capturing
ultrasound video data to assess cardiac structure and function. Artificial
intelligence (AI) in echocardiography has the potential to streamline manual
tasks and improve reproducibility and precision. However, most echocardiography
AI models are single-view, single-task systems that do not synthesize
complementary information from multiple views captured during a full exam, and
thus lead to limited performance and scope of applications. To address this
problem, we introduce EchoPrime, a multi-view, view-informed, video-based
vision-language foundation model trained on over 12 million video-report pairs.
EchoPrime uses contrastive learning to train a unified embedding model for all
standard views in a comprehensive echocardiogram study with representation of
both rare and common diseases and diagnoses. EchoPrime then utilizes
view-classification and a view-informed anatomic attention model to weight
video-specific interpretations that accurately maps the relationship between
echocardiographic views and anatomical structures. With retrieval-augmented
interpretation, EchoPrime integrates information from all echocardiogram videos
in a comprehensive study and performs holistic comprehensive clinical
echocardiography interpretation. In datasets from two independent healthcare
systems, EchoPrime achieves state-of-the art performance on 23 diverse
benchmarks of cardiac form and function, surpassing the performance of both
task-specific approaches and prior foundation models. Following rigorous
clinical evaluation, EchoPrime can assist physicians in the automated
preliminary assessment of comprehensive echocardiography.
EchoPrime, a multi-view, view-informed, vision-language foundation model, leverages contrastive learning and view-classification to enhance the interpretation of echocardiogram videos, outperforming previous models and assisting in clinical echocardiography.
contrastive learning unified embedding model view-classification anatomic attention model retrieval-augmented interpretation echocardiogram videos cardiac form and function