ORID: Organ-Regional Information Driven Framework for Radiology Report
  Generation
The objective of Radiology Report Generation (RRG) is to automatically
generate coherent textual analyses of diseases based on radiological images,
thereby alleviating the workload of radiologists. Current AI-based methods for
RRG primarily focus on modifications to the encoder-decoder model architecture.
To advance these approaches, this paper introduces an Organ-Regional
Information Driven (ORID) framework which can effectively integrate multi-modal
information and reduce the influence of noise from unrelated organs.
Specifically, based on the LLaVA-Med, we first construct an RRG-related
instruction dataset to improve organ-regional diagnosis description ability and
get the LLaVA-Med-RRG. After that, we propose an organ-based cross-modal fusion
module to effectively combine the information from the organ-regional diagnosis
description and radiology image. To further reduce the influence of noise from
unrelated organs on the radiology report generation, we introduce an organ
importance coefficient analysis module, which leverages Graph Neural Network
(GNN) to examine the interconnections of the cross-modal information of each
organ region. Extensive experiments an1d comparisons with state-of-the-art
methods across various evaluation metrics demonstrate the superior performance
of our proposed method.
673ee6ae8322f5b4acdcb5d4
Tiancheng Gu
claimed_verified
2025-04-25T08:36:41.553Z
641030c77a15af878ae5bd8f
/avatars/8a5037edf55c78ebc317c8b191343671.svg
TianchengGu
user
TianchengGu
673ee6ae8322f5b4acdcb5d5
Kaicheng Yang
claimed_verified
2024-11-21T08:09:25.357Z
63e202f352b7578dba448ab5
https://cdn-avatars.huggingface.co/v1/production/uploads/63e202f352b7578dba448ab5/8itVBLcv14m7OVsoF8h1o.jpeg
Yang
user
Kaichengalex
673ee6ae8322f5b4acdcb5d6
Xiang An
673ee6ae8322f5b4acdcb5d7
Ziyong Feng
673ee6ae8322f5b4acdcb5d8
Dongnan Liu
admin_assigned
2024-11-22T13:23:16.931Z
65e7f19e14856e8859fd8adc
/avatars/203e919c361db94028a1b3c6ea52f0c2.svg
Dongnan Liu
user
Nina0607
673ee6ae8322f5b4acdcb5d9
Weidong Cai
ORID: Organ-Regional Information Driven Framework for Radiology Report
  Generation
The objective of Radiology Report Generation (RRG) is to automatically
generate coherent textual analyses of diseases based on radiological images,
thereby alleviating the workload of radiologists. Current AI-based methods for
RRG primarily focus on modifications to the encoder-decoder model architecture.
To advance these approaches, this paper introduces an Organ-Regional
Information Driven (ORID) framework which can effectively integrate multi-modal
information and reduce the influence of noise from unrelated organs.
Specifically, based on the LLaVA-Med, we first construct an RRG-related
instruction dataset to improve organ-regional diagnosis description ability and
get the LLaVA-Med-RRG. After that, we propose an organ-based cross-modal fusion
module to effectively combine the information from the organ-regional diagnosis
description and radiology image. To further reduce the influence of noise from
unrelated organs on the radiology report generation, we introduce an organ
importance coefficient analysis module, which leverages Graph Neural Network
(GNN) to examine the interconnections of the cross-modal information of each
organ region. Extensive experiments an1d comparisons with state-of-the-art
methods across various evaluation metrics demonstrate the superior performance
of our proposed method.
The ORID framework improves radiology report generation by integrating organ-specific information using LLaVA-Med and Graph Neural Networks, reducing noise from unrelated organs and enhancing performance.
LaLaVA-Med encoder-decoder model Organ-Regional Information Driven (ORID) framework organ-based cross-modal fusion module organ importance coefficient analysis module Graph Neural Network (GNN)