Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal
  Evolution of Human States
As Large Language Models (LLMs) increasingly participate in human-AI
interactions, evaluating their Theory of Mind (ToM) capabilities - particularly
their ability to track dynamic mental states - becomes crucial. While existing
benchmarks assess basic ToM abilities, they predominantly focus on static
snapshots of mental states, overlooking the temporal evolution that
characterizes real-world social interactions. We present DynToM, a
novel benchmark specifically designed to evaluate LLMs' ability to understand
and track the temporal progression of mental states across interconnected
scenarios. Through a systematic four-step framework, we generate 1,100 social
contexts encompassing 5,500 scenarios and 78,100 questions, each validated for
realism and quality. Our comprehensive evaluation of ten state-of-the-art LLMs
reveals that their average performance underperforms humans by 44.7\%, with
performance degrading significantly when tracking and reasoning about the shift
of mental states. This performance gap highlights fundamental limitations in
current LLMs' ability to model the dynamic nature of human mental states.
6833c6ff97966d18e7b995b0
Yang Xiao
claimed_verified
2025-05-28T19:14:41.639Z
6002c316698168af3bb9f4a6
https://cdn-avatars.huggingface.co/v1/production/uploads/6002c316698168af3bb9f4a6/M2J2QFCRc5RhYRoz7OuZN.png
yangxiao
user
YangXiao-nlp
6833c6ff97966d18e7b995b1
Jiashuo Wang
6833c6ff97966d18e7b995b2
Qiancheng Xu
6833c6ff97966d18e7b995b3
Changhe Song
6833c6ff97966d18e7b995b4
Chunpu Xu
6833c6ff97966d18e7b995b5
Yi Cheng
6833c6ff97966d18e7b995b6
Wenjie Li
6833c6ff97966d18e7b995b7
Pengfei Liu
claimed_verified
2025-06-29T10:14:28.993Z
6144a0c4ff1146bbd84d9865
https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
Pengfei Liu
user
Pengfei
Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal
  Evolution of Human States
As Large Language Models (LLMs) increasingly participate in human-AI
interactions, evaluating their Theory of Mind (ToM) capabilities - particularly
their ability to track dynamic mental states - becomes crucial. While existing
benchmarks assess basic ToM abilities, they predominantly focus on static
snapshots of mental states, overlooking the temporal evolution that
characterizes real-world social interactions. We present DynToM, a
novel benchmark specifically designed to evaluate LLMs' ability to understand
and track the temporal progression of mental states across interconnected
scenarios. Through a systematic four-step framework, we generate 1,100 social
contexts encompassing 5,500 scenarios and 78,100 questions, each validated for
realism and quality. Our comprehensive evaluation of ten state-of-the-art LLMs
reveals that their average performance underperforms humans by 44.7\%, with
performance degrading significantly when tracking and reasoning about the shift
of mental states. This performance gap highlights fundamental limitations in
current LLMs' ability to model the dynamic nature of human mental states.
The DynToM benchmark evaluates LLMs' ability to track and understand the temporal progression of mental states, revealing significant gaps compared to human performance.
Large Language Models (LLMs) Theory of Mind (ToM) DynToM social contexts mental states temporal progression evaluation framework state-of-the-art LLMs performance gap dynamic mental states