Medical large language models are easily distracted
Large language models (LLMs) have the potential to transform medicine, but
real-world clinical scenarios contain extraneous information that can hinder
performance. The rise of assistive technologies like ambient dictation, which
automatically generates draft notes from live patient encounters, has the
potential to introduce additional noise making it crucial to assess the ability
of LLM's to filter relevant data. To investigate this, we developed
MedDistractQA, a benchmark using USMLE-style questions embedded with simulated
real-world distractions. Our findings show that distracting statements
(polysemous words with clinical meanings used in a non-clinical context or
references to unrelated health conditions) can reduce LLM accuracy by up to
17.9%. Commonly proposed solutions to improve model performance such as
retrieval-augmented generation (RAG) and medical fine-tuning did not change
this effect and in some cases introduced their own confounders and further
degraded performance. Our findings suggest that LLMs natively lack the logical
mechanisms necessary to distinguish relevant from irrelevant clinical
information, posing challenges for real-world applications. MedDistractQA and
our results highlights the need for robust mitigation strategies to enhance LLM
resilience to extraneous information.
67ee95f48b0b5419e49c37dc
Krithik Vishwanath
extracted_pending
2025-04-03T14:06:45.697Z
66a02bb71bb371390c4ae519
/avatars/2c4791516aae0b20a9c9ce0542dc966e.svg
Krithik Vishwanath
user
KrithikV
67ee95f48b0b5419e49c37dd
Anton Alyakin
67ee95f48b0b5419e49c37de
Daniel Alexander Alber
67ee95f48b0b5419e49c37df
Jin Vivian Lee
67ee95f48b0b5419e49c37e0
Douglas Kondziolka
67ee95f48b0b5419e49c37e1
Eric Karl Oermann
Medical large language models are easily distracted
Large language models (LLMs) have the potential to transform medicine, but
real-world clinical scenarios contain extraneous information that can hinder
performance. The rise of assistive technologies like ambient dictation, which
automatically generates draft notes from live patient encounters, has the
potential to introduce additional noise making it crucial to assess the ability
of LLM's to filter relevant data. To investigate this, we developed
MedDistractQA, a benchmark using USMLE-style questions embedded with simulated
real-world distractions. Our findings show that distracting statements
(polysemous words with clinical meanings used in a non-clinical context or
references to unrelated health conditions) can reduce LLM accuracy by up to
17.9%. Commonly proposed solutions to improve model performance such as
retrieval-augmented generation (RAG) and medical fine-tuning did not change
this effect and in some cases introduced their own confounders and further
degraded performance. Our findings suggest that LLMs natively lack the logical
mechanisms necessary to distinguish relevant from irrelevant clinical
information, posing challenges for real-world applications. MedDistractQA and
our results highlights the need for robust mitigation strategies to enhance LLM
resilience to extraneous information.
MedDistractQA benchmark reveals LLMs struggle with filtering relevant clinical information amidst distractions, challenging their real-world applicability.
LLMs assistive technologies ambient dictation MedDistractQA USMLE-style questions distracting statements polysemous words clinical meanings retrieval-augmented generation medical fine-tuning logical mechanisms robust mitigation strategies