How FaR Are Large Language Models From Agents with Theory-of-Mind?
"Thinking is for Doing." Humans can infer other people's mental states from
observations--an ability called Theory-of-Mind (ToM)--and subsequently act
pragmatically on those inferences. Existing question answering benchmarks such
as ToMi ask models questions to make inferences about beliefs of characters in
a story, but do not test whether models can then use these inferences to guide
their actions. We propose a new evaluation paradigm for large language models
(LLMs): Thinking for Doing (T4D), which requires models to connect inferences
about others' mental states to actions in social scenarios. Experiments on T4D
demonstrate that LLMs such as GPT-4 and PaLM 2 seemingly excel at tracking
characters' beliefs in stories, but they struggle to translate this capability
into strategic action. Our analysis reveals the core challenge for LLMs lies in
identifying the implicit inferences about mental states without being
explicitly asked about as in ToMi, that lead to choosing the correct action in
T4D. To bridge this gap, we introduce a zero-shot prompting framework, Foresee
and Reflect (FaR), which provides a reasoning structure that encourages LLMs to
anticipate future challenges and reason about potential actions. FaR boosts
GPT-4's performance from 50% to 71% on T4D, outperforming other prompting
methods such as Chain-of-Thought and Self-Ask. Moreover, FaR generalizes to
diverse out-of-distribution story structures and scenarios that also require
ToM inferences to choose an action, consistently outperforming other methods
including few-shot in-context learning.
651f60b4f23fc101b16196c1
Pei Zhou
extracted_pending
2023-10-06T01:19:51.694Z
60844fd056f7623a0b84914b
/avatars/f7d2763258b8c614783ead9480fbcf8d.svg
Pei Zhou
user
peiz
651f60b4f23fc101b16196c2
Aman Madaan
admin_assigned
2023-10-09T08:32:58.731Z
61bbc2b2d96f5d4111901dcb
/avatars/83c826d5cad7bab27d33585cb7e64575.svg
Aman Madaan
user
amanmadaan
651f60b4f23fc101b16196c3
Srividya Pranavi Potharaju
651f60b4f23fc101b16196c4
Aditya Gupta
claimed_verified
2023-10-21T09:22:55.725Z
64bac8eb01f1983a862e58b2
/avatars/ba005db511bf9fa2e79ca2e78852f8fd.svg
Aditya Gupta
user
aditya2211
651f60b4f23fc101b16196c5
Kevin R. McKee
admin_assigned
2023-10-09T08:33:40.074Z
624911a1edf42bf849d0ffc5
/avatars/fdf70a412b2b90d0407dc6adb484319f.svg
Kevin Mckee
user
Kmckee95
651f60b4f23fc101b16196c6
Ari Holtzman
claimed_verified
2023-10-11T07:34:49.773Z
62a930029e44ab4160814c07
/avatars/16eafa37a14715dd286d2670af88a0d0.svg
Ari Holtzman
user
ari-holtzman
651f60b4f23fc101b16196c7
Jay Pujara
651f60b4f23fc101b16196c8
Xiang Ren
651f60b4f23fc101b16196c9
Swaroop Mishra
admin_assigned
2023-10-09T08:35:19.039Z
61841feb7c380c66d40cad54
/avatars/7a79362a24b844860781fb0625527eae.svg
Swaroop Mishra
user
swaroop7
651f60b4f23fc101b16196ca
Aida Nematzadeh
651f60b4f23fc101b16196cb
Shyam Upadhyay
651f60b4f23fc101b16196cc
Manaal Faruqui
How FaR Are Large Language Models From Agents with Theory-of-Mind?
"Thinking is for Doing." Humans can infer other people's mental states from
observations--an ability called Theory-of-Mind (ToM)--and subsequently act
pragmatically on those inferences. Existing question answering benchmarks such
as ToMi ask models questions to make inferences about beliefs of characters in
a story, but do not test whether models can then use these inferences to guide
their actions. We propose a new evaluation paradigm for large language models
(LLMs): Thinking for Doing (T4D), which requires models to connect inferences
about others' mental states to actions in social scenarios. Experiments on T4D
demonstrate that LLMs such as GPT-4 and PaLM 2 seemingly excel at tracking
characters' beliefs in stories, but they struggle to translate this capability
into strategic action. Our analysis reveals the core challenge for LLMs lies in
identifying the implicit inferences about mental states without being
explicitly asked about as in ToMi, that lead to choosing the correct action in
T4D. To bridge this gap, we introduce a zero-shot prompting framework, Foresee
and Reflect (FaR), which provides a reasoning structure that encourages LLMs to
anticipate future challenges and reason about potential actions. FaR boosts
GPT-4's performance from 50% to 71% on T4D, outperforming other prompting
methods such as Chain-of-Thought and Self-Ask. Moreover, FaR generalizes to
diverse out-of-distribution story structures and scenarios that also require
ToM inferences to choose an action, consistently outperforming other methods
including few-shot in-context learning.
LLMs perform well in inferring beliefs but struggle translating these into actions; Foresee and Reflect (FaR) improves their action guidance in social scenarios.
Theory-of-Mind (ToM) large language models (LLMs) Thinking for Doing (T4D) mental states strategic action zero-shot prompting Chain-of-Thought Self-Ask