Think in Games: Learning to Reason in Games via Reinforcement Learning
  with Large Language Models
Large language models (LLMs) excel at complex reasoning tasks such as
mathematics and coding, yet they frequently struggle with simple interactive
tasks that young children perform effortlessly. This discrepancy highlights a
critical gap between declarative knowledge (knowing about something) and
procedural knowledge (knowing how to do something). Although traditional
reinforcement learning (RL) agents can acquire procedural knowledge through
environmental interaction, they often operate as black boxes and require
substantial training data. In contrast, LLMs possess extensive world knowledge
and reasoning capabilities, but are unable to effectively convert this static
knowledge into dynamic decision-making in interactive settings. To address this
challenge, we propose Think in Games (TiG), a novel framework that empowers
LLMs to develop procedural understanding through direct interaction with game
environments, while retaining their inherent reasoning and explanatory
abilities. Specifically, TiG reformulates RL-based decision-making as a
language modeling task: LLMs generate language-guided policies, which are
refined iteratively through online reinforcement learning based on
environmental feedback. Our experimental results show that TiG successfully
bridges the gap between declarative and procedural knowledge, achieving
competitive performance with dramatically lower data and computational demands
compared to conventional RL methods. Moreover, TiG provides step-by-step
natural language explanations for its decisions, greatly improving transparency
and interpretability in complex interactive tasks.
68b51815851c6e7b001eca9f
Yi Liao
68b51815851c6e7b001ecaa0
Yu Gu
68b51815851c6e7b001ecaa1
Yuan Sui
68b51815851c6e7b001ecaa2
Zining Zhu
68b51815851c6e7b001ecaa3
Yifan Lu
68b51815851c6e7b001ecaa4
Guohua Tang
68b51815851c6e7b001ecaa5
Zhongqian Sun
68b51815851c6e7b001ecaa6
Wei Yang
Think in Games: Learning to Reason in Games via Reinforcement Learning
  with Large Language Models
Large language models (LLMs) excel at complex reasoning tasks such as
mathematics and coding, yet they frequently struggle with simple interactive
tasks that young children perform effortlessly. This discrepancy highlights a
critical gap between declarative knowledge (knowing about something) and
procedural knowledge (knowing how to do something). Although traditional
reinforcement learning (RL) agents can acquire procedural knowledge through
environmental interaction, they often operate as black boxes and require
substantial training data. In contrast, LLMs possess extensive world knowledge
and reasoning capabilities, but are unable to effectively convert this static
knowledge into dynamic decision-making in interactive settings. To address this
challenge, we propose Think in Games (TiG), a novel framework that empowers
LLMs to develop procedural understanding through direct interaction with game
environments, while retaining their inherent reasoning and explanatory
abilities. Specifically, TiG reformulates RL-based decision-making as a
language modeling task: LLMs generate language-guided policies, which are
refined iteratively through online reinforcement learning based on
environmental feedback. Our experimental results show that TiG successfully
bridges the gap between declarative and procedural knowledge, achieving
competitive performance with dramatically lower data and computational demands
compared to conventional RL methods. Moreover, TiG provides step-by-step
natural language explanations for its decisions, greatly improving transparency
and interpretability in complex interactive tasks.
Think in Games (TiG) framework enables large language models to develop procedural knowledge through interactive game environments, achieving competitive performance with reduced data and computational demands while providing transparent explanations.
large language models LLMs complex reasoning tasks declarative knowledge procedural knowledge reinforcement learning RL environmental interaction black boxes training data world knowledge reasoning capabilities language modeling task language-guided policies online reinforcement learning environmental feedback competitive performance data demands computational demands step-by-step natural language explanations transparency interpretability