Clinical Text Summarization: Adapting Large Language Models Can
  Outperform Human Experts
Sifting through vast textual data and summarizing key information imposes a
substantial burden on how clinicians allocate their time. Although large
language models (LLMs) have shown immense promise in natural language
processing (NLP) tasks, their efficacy across diverse clinical summarization
tasks has not yet been rigorously examined. In this work, we employ domain
adaptation methods on eight LLMs, spanning six datasets and four distinct
summarization tasks: radiology reports, patient questions, progress notes, and
doctor-patient dialogue. Our thorough quantitative assessment reveals
trade-offs between models and adaptation methods in addition to instances where
recent advances in LLMs may not lead to improved results. Further, in a
clinical reader study with six physicians, we depict that summaries from the
best adapted LLM are preferable to human summaries in terms of completeness and
correctness. Our ensuing qualitative analysis delineates mutual challenges
faced by both LLMs and human experts. Lastly, we correlate traditional
quantitative NLP metrics with reader study scores to enhance our understanding
of how these metrics align with physician preferences. Our research marks the
first evidence of LLMs outperforming human experts in clinical text
summarization across multiple tasks. This implies that integrating LLMs into
clinical workflows could alleviate documentation burden, empowering clinicians
to focus more on personalized patient care and other irreplaceable human
aspects of medicine.
6503bd65c6ce94dc0e1dd13c
Dave Van Veen
extracted_confirmed
2023-09-15T05:59:55.991Z
64e36dd9618cd90997e048f2
/avatars/2a239bed193982596114d52afea1bb97.svg
Dave Van Veen
user
davevanveen
6503bd65c6ce94dc0e1dd13d
Cara Van Uden
admin_assigned
2023-09-15T07:57:13.219Z
64c7c84fe761f47061035533
/avatars/810b73fd2ca0f6e7028a84496e542e91.svg
Cara Van Uden
user
cvanuden
6503bd65c6ce94dc0e1dd13e
Louis Blankemeier
admin_assigned
2023-09-15T07:53:03.851Z
6377044ea3b787faca443cc0
/avatars/5cd6cc70ad597dd83776018a452947d0.svg
Louis Blankemeier
user
louisblankemeier
6503bd65c6ce94dc0e1dd13f
Jean-Benoit Delbrouck
claimed_verified
2024-07-18T09:10:58.982Z
62716952bcef985363db8485
https://cdn-avatars.huggingface.co/v1/production/uploads/62716952bcef985363db8485/zJPPo5xlwZRJdEuwYsYKp.jpeg
JB D.
user
IAMJB
6503bd65c6ce94dc0e1dd140
Asad Aali
admin_assigned
2023-09-15T07:57:59.027Z
64bac7c5e38420aaba8ea197
https://cdn-avatars.huggingface.co/v1/production/uploads/64bac7c5e38420aaba8ea197/HKLAvXe43eDK3qqNc4aXj.png
Asad Aali
user
asadaali
6503bd65c6ce94dc0e1dd141
Christian Bluethgen
admin_assigned
2023-09-15T07:54:08.286Z
60c540b2fcb103f4f6a53c5a
https://cdn-avatars.huggingface.co/v1/production/uploads/1653944246103-60c540b2fcb103f4f6a53c5a.jpeg
Christian Bluethgen
user
pogchamp
6503bd65c6ce94dc0e1dd142
Anuj Pareek
6503bd65c6ce94dc0e1dd143
Malgorzata Polacin
6503bd65c6ce94dc0e1dd144
William Collins
admin_assigned
2023-09-15T08:00:55.986Z
642fd225450c0de9a1d1b641
/avatars/7b457f909e94cda3875884381bcd8e39.svg
William Collins
user
wwcollins
6503bd65c6ce94dc0e1dd145
Neera Ahuja
6503bd65c6ce94dc0e1dd146
Curtis P. Langlotz
admin_assigned
2023-09-15T07:55:50.293Z
62867edd1d99648808888d7b
/avatars/22d764782c5169634deec6f43ece7500.svg
Curt Langlotz
user
cplanglotz
6503bd65c6ce94dc0e1dd147
Jason Hom
6503bd65c6ce94dc0e1dd148
Sergios Gatidis
6503bd65c6ce94dc0e1dd149
John Pauly
6503bd65c6ce94dc0e1dd14a
Akshay S. Chaudhari
admin_assigned
2023-09-15T07:57:03.094Z
6236533b76c8a780323af640
/avatars/18078ad26aa44312a4927160216e5943.svg
Akshay Chaudhari
user
akshaysc
Clinical Text Summarization: Adapting Large Language Models Can
  Outperform Human Experts
Sifting through vast textual data and summarizing key information imposes a
substantial burden on how clinicians allocate their time. Although large
language models (LLMs) have shown immense promise in natural language
processing (NLP) tasks, their efficacy across diverse clinical summarization
tasks has not yet been rigorously examined. In this work, we employ domain
adaptation methods on eight LLMs, spanning six datasets and four distinct
summarization tasks: radiology reports, patient questions, progress notes, and
doctor-patient dialogue. Our thorough quantitative assessment reveals
trade-offs between models and adaptation methods in addition to instances where
recent advances in LLMs may not lead to improved results. Further, in a
clinical reader study with six physicians, we depict that summaries from the
best adapted LLM are preferable to human summaries in terms of completeness and
correctness. Our ensuing qualitative analysis delineates mutual challenges
faced by both LLMs and human experts. Lastly, we correlate traditional
quantitative NLP metrics with reader study scores to enhance our understanding
of how these metrics align with physician preferences. Our research marks the
first evidence of LLMs outperforming human experts in clinical text
summarization across multiple tasks. This implies that integrating LLMs into
clinical workflows could alleviate documentation burden, empowering clinicians
to focus more on personalized patient care and other irreplaceable human
aspects of medicine.
Domain-adapted large language models outperform human experts in various clinical text summarization tasks, improving completeness, correctness, and aligning with physician preferences.
large language models domain adaptation natural language processing clinical summarization radiology reports patient questions progress notes doctor-patient dialogue clinical reader study traditional quantitative NLP metrics