Agents Play Thousands of 3D Video Games
We present PORTAL, a novel framework for developing artificial intelligence
agents capable of playing thousands of 3D video games through language-guided
policy generation. By transforming decision-making problems into language
modeling tasks, our approach leverages large language models (LLMs) to generate
behavior trees represented in domain-specific language (DSL). This method
eliminates the computational burden associated with traditional reinforcement
learning approaches while preserving strategic depth and rapid adaptability.
Our framework introduces a hybrid policy structure that combines rule-based
nodes with neural network components, enabling both high-level strategic
reasoning and precise low-level control. A dual-feedback mechanism
incorporating quantitative game metrics and vision-language model analysis
facilitates iterative policy improvement at both tactical and strategic levels.
The resulting policies are instantaneously deployable, human-interpretable, and
capable of generalizing across diverse gaming environments. Experimental
results demonstrate PORTAL's effectiveness across thousands of first-person
shooter (FPS) games, showcasing significant improvements in development
efficiency, policy generalization, and behavior diversity compared to
traditional approaches. PORTAL represents a significant advancement in game AI
development, offering a practical solution for creating sophisticated agents
that can operate across thousands of commercial video games with minimal
development overhead. Experiment results on the 3D video games are best viewed
on https://zhongwen.one/projects/portal .
67dcfafa73e582f950d360f3
Zhongwen Xu
claimed_verified
2025-03-21T11:40:17.259Z
672db76fa34d64e774fc42c9
/avatars/be529f4bfcc40514697facbe8e874735.svg
Zhongwen Xu
user
zhongwenxu
67dcfafa73e582f950d360f4
Xianliang Wang
67dcfafa73e582f950d360f5
Siyi Li
67dcfafa73e582f950d360f6
Tao Yu
67dcfafa73e582f950d360f7
Liang Wang
67dcfafa73e582f950d360f8
Qiang Fu
67dcfafa73e582f950d360f9
Wei Yang
Agents Play Thousands of 3D Video Games
We present PORTAL, a novel framework for developing artificial intelligence
agents capable of playing thousands of 3D video games through language-guided
policy generation. By transforming decision-making problems into language
modeling tasks, our approach leverages large language models (LLMs) to generate
behavior trees represented in domain-specific language (DSL). This method
eliminates the computational burden associated with traditional reinforcement
learning approaches while preserving strategic depth and rapid adaptability.
Our framework introduces a hybrid policy structure that combines rule-based
nodes with neural network components, enabling both high-level strategic
reasoning and precise low-level control. A dual-feedback mechanism
incorporating quantitative game metrics and vision-language model analysis
facilitates iterative policy improvement at both tactical and strategic levels.
The resulting policies are instantaneously deployable, human-interpretable, and
capable of generalizing across diverse gaming environments. Experimental
results demonstrate PORTAL's effectiveness across thousands of first-person
shooter (FPS) games, showcasing significant improvements in development
efficiency, policy generalization, and behavior diversity compared to
traditional approaches. PORTAL represents a significant advancement in game AI
development, offering a practical solution for creating sophisticated agents
that can operate across thousands of commercial video games with minimal
development overhead. Experiment results on the 3D video games are best viewed
on https://zhongwen.one/projects/portal .
A framework that uses large language models to generate behavior trees for AI agents to play 3D video games, combining rule-based and neural network components with a dual-feedback mechanism.
large language models behavior trees domain-specific language hybrid policy structure dual-feedback mechanism vision-language model first-person shooter game AI development behavior diversity