Towards Conversational Diagnostic AI
At the heart of medicine lies the physician-patient dialogue, where skillful
history-taking paves the way for accurate diagnosis, effective management, and
enduring trust. Artificial Intelligence (AI) systems capable of diagnostic
dialogue could increase accessibility, consistency, and quality of care.
However, approximating clinicians' expertise is an outstanding grand challenge.
Here, we introduce AMIE (Articulate Medical Intelligence Explorer), a Large
Language Model (LLM) based AI system optimized for diagnostic dialogue.
  AMIE uses a novel self-play based simulated environment with automated
feedback mechanisms for scaling learning across diverse disease conditions,
specialties, and contexts. We designed a framework for evaluating
clinically-meaningful axes of performance including history-taking, diagnostic
accuracy, management reasoning, communication skills, and empathy. We compared
AMIE's performance to that of primary care physicians (PCPs) in a randomized,
double-blind crossover study of text-based consultations with validated patient
actors in the style of an Objective Structured Clinical Examination (OSCE). The
study included 149 case scenarios from clinical providers in Canada, the UK,
and India, 20 PCPs for comparison with AMIE, and evaluations by specialist
physicians and patient actors. AMIE demonstrated greater diagnostic accuracy
and superior performance on 28 of 32 axes according to specialist physicians
and 24 of 26 axes according to patient actors. Our research has several
limitations and should be interpreted with appropriate caution. Clinicians were
limited to unfamiliar synchronous text-chat which permits large-scale
LLM-patient interactions but is not representative of usual clinical practice.
While further research is required before AMIE could be translated to
real-world settings, the results represent a milestone towards conversational
diagnostic AI.
65a0a488ff65c2d46d9efe22
Tao Tu
65a0a488ff65c2d46d9efe23
Anil Palepu
65a0a488ff65c2d46d9efe24
Mike Schaekermann
admin_assigned
2024-01-12T10:09:36.391Z
64ebb75f171e6c9862d261c9
/avatars/b7750f5d6281f906cfa5c717a44e2e13.svg
Mike Schaekermann
user
mikeschaekermann
65a0a488ff65c2d46d9efe25
Khaled Saab
65a0a488ff65c2d46d9efe26
Jan Freyberg
admin_assigned
2024-01-12T10:09:47.258Z
629656557d586f66c39206e7
/avatars/91245f3b6a2eccebef872ed00bdfea73.svg
Jan Freyberg
user
jfreyberg
65a0a488ff65c2d46d9efe27
Ryutaro Tanno
65a0a488ff65c2d46d9efe28
Amy Wang
65a0a488ff65c2d46d9efe29
Brenna Li
65a0a488ff65c2d46d9efe2a
Mohamed Amin
65a0a488ff65c2d46d9efe2b
Nenad Tomasev
65a0a488ff65c2d46d9efe2c
Shekoofeh Azizi
65a0a488ff65c2d46d9efe2d
Karan Singhal
admin_assigned
2024-01-12T10:10:19.355Z
654bb16a48b474120258371c
/avatars/7ad4a33503533ddbb54c71b634fda89b.svg
Karan Singhal
user
karan1149
65a0a488ff65c2d46d9efe2e
Yong Cheng
65a0a488ff65c2d46d9efe2f
Le Hou
admin_assigned
2024-01-12T10:10:44.869Z
626b2402c804c432c1b5004c
/avatars/a119644422335e260af0ca4ef6399870.svg
Le Hou
user
solooobear
65a0a488ff65c2d46d9efe30
Albert Webson
admin_assigned
2024-01-12T10:10:52.013Z
617ada70ff3db6021d069798
/avatars/f14c59fd56a49dd9a925285a4fb66fff.svg
Albert Webson
user
awebson
65a0a488ff65c2d46d9efe31
Kavita Kulkarni
65a0a488ff65c2d46d9efe32
S Sara Mahdavi
65a0a488ff65c2d46d9efe33
Christopher Semturs
65a0a488ff65c2d46d9efe34
Juraj Gottweis
admin_assigned
2024-01-12T10:11:10.554Z
6381e100cae34727b9ca33ce
/avatars/f4e047e82e41ec507c6f51aef7560d87.svg
Juraj Gottweis
user
juro
65a0a488ff65c2d46d9efe35
Joelle Barral
65a0a488ff65c2d46d9efe36
Katherine Chou
65a0a488ff65c2d46d9efe37
Greg S Corrado
65a0a488ff65c2d46d9efe38
Yossi Matias
65a0a488ff65c2d46d9efe39
Alan Karthikesalingam
65a0a488ff65c2d46d9efe3a
Vivek Natarajan
admin_assigned
2024-01-12T10:11:37.514Z
617a6af2941993035fbaf29c
/avatars/a389671494782f06ff8502c934090cc0.svg
Vivek Natarajan
user
vivnat
Towards Conversational Diagnostic AI
At the heart of medicine lies the physician-patient dialogue, where skillful
history-taking paves the way for accurate diagnosis, effective management, and
enduring trust. Artificial Intelligence (AI) systems capable of diagnostic
dialogue could increase accessibility, consistency, and quality of care.
However, approximating clinicians' expertise is an outstanding grand challenge.
Here, we introduce AMIE (Articulate Medical Intelligence Explorer), a Large
Language Model (LLM) based AI system optimized for diagnostic dialogue.
  AMIE uses a novel self-play based simulated environment with automated
feedback mechanisms for scaling learning across diverse disease conditions,
specialties, and contexts. We designed a framework for evaluating
clinically-meaningful axes of performance including history-taking, diagnostic
accuracy, management reasoning, communication skills, and empathy. We compared
AMIE's performance to that of primary care physicians (PCPs) in a randomized,
double-blind crossover study of text-based consultations with validated patient
actors in the style of an Objective Structured Clinical Examination (OSCE). The
study included 149 case scenarios from clinical providers in Canada, the UK,
and India, 20 PCPs for comparison with AMIE, and evaluations by specialist
physicians and patient actors. AMIE demonstrated greater diagnostic accuracy
and superior performance on 28 of 32 axes according to specialist physicians
and 24 of 26 axes according to patient actors. Our research has several
limitations and should be interpreted with appropriate caution. Clinicians were
limited to unfamiliar synchronous text-chat which permits large-scale
LLM-patient interactions but is not representative of usual clinical practice.
While further research is required before AMIE could be translated to
real-world settings, the results represent a milestone towards conversational
diagnostic AI.
An AI system named AMIE, designed with a large language model optimized for diagnostic dialogue, demonstrated better performance than primary care physicians in several clinical axes through a simulated environment and textual consultations with patient actors.
Large Language Model self-play diagnostic dialogue history-taking diagnostic accuracy management reasoning communication skills empathy Objective Structured Clinical Examination textual consultations conversational diagnostic AI