μ^2Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for
  Radiology Report Generation
Automated radiology report generation (RRG) aims to produce detailed textual
reports from clinical imaging, such as computed tomography (CT) scans, to
improve the accuracy and efficiency of diagnosis and provision of management
advice. RRG is complicated by two key challenges: (1) inherent complexity in
extracting relevant information from imaging data under resource constraints,
and (2) difficulty in objectively evaluating discrepancies between
model-generated and expert-written reports. To address these challenges, we
propose mu^2LLM, a textbf{mu}ltiscale
textbf{mu}ltimodal large language models for RRG tasks. The
novel {mu}^2Tokenizer, as an intermediate layer, integrates multi-modal
features from the multiscale visual tokenizer and the text tokenizer, then
enhances report generation quality through direct preference optimization
(DPO), guided by GREEN-RedLlama. Experimental results on four large CT
image-report medical datasetdemonstrate that our method outperforms existing
approaches, highlighting the potential of our fine-tuned mu^2LLMs on limited
data for RRG tasks.
68653e95d706a6e48024af9b
Siyou Li
claimed_verified
2025-07-03T07:32:44.870Z
62d53e556dff888212d58724
https://cdn-avatars.huggingface.co/v1/production/uploads/62d53e556dff888212d58724/kJxZGBvR0iMnmGC890g-e.png
Siyou Li
user
SiyouLi
68653e95d706a6e48024af9c
Pengyao Qin
68653e95d706a6e48024af9d
Huanan Wu
68653e95d706a6e48024af9e
Dong Nie
68653e95d706a6e48024af9f
Arun J. Thirunavukarasu
68653e95d706a6e48024afa0
Juntao Yu
68653e95d706a6e48024afa1
Le Zhang
claimed_verified
2025-07-03T16:19:10.289Z
6866735fcb2dbd91988e8bed
/avatars/afa97d6e209d107ab01501fb4510ae08.svg
Le Zhang
user
thisislez
μ^2Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for
  Radiology Report Generation
Automated radiology report generation (RRG) aims to produce detailed textual
reports from clinical imaging, such as computed tomography (CT) scans, to
improve the accuracy and efficiency of diagnosis and provision of management
advice. RRG is complicated by two key challenges: (1) inherent complexity in
extracting relevant information from imaging data under resource constraints,
and (2) difficulty in objectively evaluating discrepancies between
model-generated and expert-written reports. To address these challenges, we
propose mu^2LLM, a textbf{mu}ltiscale
textbf{mu}ltimodal large language models for RRG tasks. The
novel {mu}^2Tokenizer, as an intermediate layer, integrates multi-modal
features from the multiscale visual tokenizer and the text tokenizer, then
enhances report generation quality through direct preference optimization
(DPO), guided by GREEN-RedLlama. Experimental results on four large CT
image-report medical datasetdemonstrate that our method outperforms existing
approaches, highlighting the potential of our fine-tuned mu^2LLMs on limited
data for RRG tasks.
A multiscale multimodal large language model, $\mu^2$LLM, enhances automated radiology report generation by integrating visual and textual features and optimizing report quality through direct preference optimization.
multiscale multimodal large language models $\mu^2$Tokenizer direct preference optimization DPO GREEN-RedLlama radiology report generation CT image-report datasets