InMind: Evaluating LLMs in Capturing and Applying Individual Human
  Reasoning Styles
LLMs have shown strong performance on human-centric reasoning tasks. While
previous evaluations have explored whether LLMs can infer intentions or detect
deception, they often overlook the individualized reasoning styles that
influence how people interpret and act in social contexts. Social deduction
games (SDGs) provide a natural testbed for evaluating individualized reasoning
styles, where different players may adopt diverse but contextually valid
reasoning strategies under identical conditions. To address this, we introduce
InMind, a cognitively grounded evaluation framework designed to assess whether
LLMs can capture and apply personalized reasoning styles in SDGs. InMind
enhances structured gameplay data with round-level strategy traces and
post-game reflections, collected under both Observer and Participant modes. It
supports four cognitively motivated tasks that jointly evaluate both static
alignment and dynamic adaptation. As a case study, we apply InMind to the game
Avalon, evaluating 11 state-of-the-art LLMs. General-purpose LLMs, even GPT-4o
frequently rely on lexical cues, struggling to anchor reflections in temporal
gameplay or adapt to evolving strategies. In contrast, reasoning-enhanced LLMs
like DeepSeek-R1 exhibit early signs of style-sensitive reasoning. These
findings reveal key limitations in current LLMs' capacity for individualized,
adaptive reasoning, and position InMind as a step toward cognitively aligned
human-AI interaction.
68abe30186b21a0e2e358b05
Zizhen Li
68abe30186b21a0e2e358b06
Chuanhao Li
68abe30186b21a0e2e358b07
Yibin Wang
68abe30186b21a0e2e358b08
Qi Chen
68abe30186b21a0e2e358b09
Diping Song
admin_assigned
2025-08-25T20:34:42.539Z
6469aa23dcbb937d56ba5077
/avatars/aa45f2dacf1c0925c5dd7614e31f9ea0.svg
Diping Song
user
dpsong
68abe30186b21a0e2e358b0a
Yukang Feng
68abe30186b21a0e2e358b0b
Jianwen Sun
68abe30186b21a0e2e358b0c
Jiaxin Ai
68abe30186b21a0e2e358b0d
Fanrui Zhang
68abe30186b21a0e2e358b0e
Mingzhu Sun
68abe30186b21a0e2e358b0f
Kaipeng Zhang
InMind: Evaluating LLMs in Capturing and Applying Individual Human
  Reasoning Styles
LLMs have shown strong performance on human-centric reasoning tasks. While
previous evaluations have explored whether LLMs can infer intentions or detect
deception, they often overlook the individualized reasoning styles that
influence how people interpret and act in social contexts. Social deduction
games (SDGs) provide a natural testbed for evaluating individualized reasoning
styles, where different players may adopt diverse but contextually valid
reasoning strategies under identical conditions. To address this, we introduce
InMind, a cognitively grounded evaluation framework designed to assess whether
LLMs can capture and apply personalized reasoning styles in SDGs. InMind
enhances structured gameplay data with round-level strategy traces and
post-game reflections, collected under both Observer and Participant modes. It
supports four cognitively motivated tasks that jointly evaluate both static
alignment and dynamic adaptation. As a case study, we apply InMind to the game
Avalon, evaluating 11 state-of-the-art LLMs. General-purpose LLMs, even GPT-4o
frequently rely on lexical cues, struggling to anchor reflections in temporal
gameplay or adapt to evolving strategies. In contrast, reasoning-enhanced LLMs
like DeepSeek-R1 exhibit early signs of style-sensitive reasoning. These
findings reveal key limitations in current LLMs' capacity for individualized,
adaptive reasoning, and position InMind as a step toward cognitively aligned
human-AI interaction.
InMind evaluates LLMs' ability to capture and apply personalized reasoning styles in social deduction games, highlighting limitations in current models' adaptive reasoning.
LLMs human-centric reasoning social deduction games SDGs reasoning styles InMind strategy traces post-game reflections Observer mode Participant mode static alignment dynamic adaptation Avalon GPT-4 DeepSeek-R1 style-sensitive reasoning cognitively aligned human-AI interaction