Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search
In this paper, we propose a new method Strategist that utilizes LLMs to
acquire new skills for playing multi-agent games through a self-improvement
process. Our method gathers quality feedback through self-play simulations with
Monte Carlo tree search and LLM-based reflection, which can then be used to
learn high-level strategic skills such as how to evaluate states that guide the
low-level execution.We showcase how our method can be used in both action
planning and dialogue generation in the context of games, achieving good
performance on both tasks. Specifically, we demonstrate that our method can
help train agents with better performance than both traditional reinforcement
learning-based approaches and other LLM-based skill learning approaches in
games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.
66c5780409ea098de4af0469
Jonathan Light
66c5780409ea098de4af046a
Min Cai
claimed_verified
2024-08-22T19:51:15.899Z
65af6d66ac291fd7fdd61bea
/avatars/5c4db3f1ece631b28c52f57525dbcb92.svg
Min Cai
user
HenryCai1129
66c5780409ea098de4af046b
Weiqin Chen
66c5780409ea098de4af046c
Guanzhi Wang
66c5780409ea098de4af046d
Xiusi Chen
66c5780409ea098de4af046e
Wei Cheng
66c5780409ea098de4af046f
Yisong Yue
66c5780409ea098de4af0470
Ziniu Hu
Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search
In this paper, we propose a new method Strategist that utilizes LLMs to
acquire new skills for playing multi-agent games through a self-improvement
process. Our method gathers quality feedback through self-play simulations with
Monte Carlo tree search and LLM-based reflection, which can then be used to
learn high-level strategic skills such as how to evaluate states that guide the
low-level execution.We showcase how our method can be used in both action
planning and dialogue generation in the context of games, achieving good
performance on both tasks. Specifically, we demonstrate that our method can
help train agents with better performance than both traditional reinforcement
learning-based approaches and other LLM-based skill learning approaches in
games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.
Strategist uses LLMs and self-play simulations to enhance agents' strategic skills in multi-agent games, outperforming traditional RL and other LLM-based methods.
LLMs self-improvement self-play simulations Monte Carlo tree search LLM-based reflection high-level strategic skills action planning dialogue generation Game of Pure Strategy The Resistance: Avalon