MedVista3D: Vision-Language Modeling for Reducing Diagnostic Errors in
  3D CT Disease Detection, Understanding and Reporting
Radiologic diagnostic errors-under-reading errors, inattentional blindness,
and communication failures-remain prevalent in clinical practice. These issues
often stem from missed localized abnormalities, limited global context, and
variability in report language. These challenges are amplified in 3D imaging,
where clinicians must examine hundreds of slices per scan. Addressing them
requires systems with precise localized detection, global volume-level
reasoning, and semantically consistent natural language reporting. However,
existing 3D vision-language models are unable to meet all three needs jointly,
lacking local-global understanding for spatial reasoning and struggling with
the variability and noise of uncurated radiology reports. We present
MedVista3D, a multi-scale semantic-enriched vision-language pretraining
framework for 3D CT analysis. To enable joint disease detection and holistic
interpretation, MedVista3D performs local and global image-text alignment for
fine-grained representation learning within full-volume context. To address
report variability, we apply language model rewrites and introduce a Radiology
Semantic Matching Bank for semantics-aware alignment. MedVista3D achieves
state-of-the-art performance on zero-shot disease classification, report
retrieval, and medical visual question answering, while transferring well to
organ segmentation and prognosis prediction. Code and datasets will be
released.
68be5223c123124955ef5fec
Yuheng Li
68be5223c123124955ef5fed
Yenho Chen
68be5223c123124955ef5fee
Yuxiang Lai
68be5223c123124955ef5fef
Jike Zhong
68be5223c123124955ef5ff0
Vanessa Wildman
68be5223c123124955ef5ff1
Xiaofeng Yang
MedVista3D: Vision-Language Modeling for Reducing Diagnostic Errors in
  3D CT Disease Detection, Understanding and Reporting
Radiologic diagnostic errors-under-reading errors, inattentional blindness,
and communication failures-remain prevalent in clinical practice. These issues
often stem from missed localized abnormalities, limited global context, and
variability in report language. These challenges are amplified in 3D imaging,
where clinicians must examine hundreds of slices per scan. Addressing them
requires systems with precise localized detection, global volume-level
reasoning, and semantically consistent natural language reporting. However,
existing 3D vision-language models are unable to meet all three needs jointly,
lacking local-global understanding for spatial reasoning and struggling with
the variability and noise of uncurated radiology reports. We present
MedVista3D, a multi-scale semantic-enriched vision-language pretraining
framework for 3D CT analysis. To enable joint disease detection and holistic
interpretation, MedVista3D performs local and global image-text alignment for
fine-grained representation learning within full-volume context. To address
report variability, we apply language model rewrites and introduce a Radiology
Semantic Matching Bank for semantics-aware alignment. MedVista3D achieves
state-of-the-art performance on zero-shot disease classification, report
retrieval, and medical visual question answering, while transferring well to
organ segmentation and prognosis prediction. Code and datasets will be
released.
MedVista3D is a multi-scale semantic-enriched vision-language pretraining framework for 3D CT analysis that addresses local-global understanding, report variability, and achieves state-of-the-art performance in disease classification, report retrieval, and medical visual question answering.
3D vision-language models local-global understanding spatial reasoning language model rewrites Radiology Semantic Matching Bank zero-shot disease classification report retrieval medical visual question answering organ segmentation prognosis prediction