Natural Language Reinforcement Learning
Reinforcement Learning (RL) mathematically formulates decision-making with
Markov Decision Process (MDP). With MDPs, researchers have achieved remarkable
breakthroughs across various domains, including games, robotics, and language
models. This paper seeks a new possibility, Natural Language Reinforcement
Learning (NLRL), by extending traditional MDP to natural language-based
representation space. Specifically, NLRL innovatively redefines RL principles,
including task objectives, policy, value function, Bellman equation, and policy
iteration, into their language counterparts. With recent advancements in large
language models (LLMs), NLRL can be practically implemented to achieve RL-like
policy and value improvement by either pure prompting or gradient-based
training. Experiments over Maze, Breakthrough, and Tic-Tac-Toe games
demonstrate the effectiveness, efficiency, and interpretability of the NLRL
framework among diverse use cases. Our code will be released at
https://github.com/waterhorse1/Natural-language-RL.
673fe3f3d2dba4a5fb919036
Xidong Feng
admin_assigned
2024-11-22T13:17:03.910Z
640c99124b5450a208ae6443
/avatars/3309f1c8f1340a93e8b7a22fb38069fa.svg
Xidong Feng
user
Waterhorse
673fe3f3d2dba4a5fb919037
Ziyu Wan
673fe3f3d2dba4a5fb919038
Haotian Fu
admin_assigned
2024-11-22T13:17:15.019Z
64c960f3d45e142a0ecdd2f4
/avatars/ccf68aea3c0fa5bcd5ca4d747405b347.svg
Haotian Fu
user
Minusadd
673fe3f3d2dba4a5fb919039
Bo Liu
claimed_verified
2024-11-22T10:02:02.835Z
635e3a76106f984574c36409
https://cdn-avatars.huggingface.co/v1/production/uploads/1667120725800-635e3a76106f984574c36409.png
Bo Liu
user
Benjamin-eecs
673fe3f3d2dba4a5fb91903a
Mengyue Yang
admin_assigned
2024-11-22T13:17:20.998Z
64bd68b0e38420aabad26a80
/avatars/fdecc96ddf7ee1809c1dfc8d0807161d.svg
Mengyue Yang
user
ymy4323460
673fe3f3d2dba4a5fb91903b
Girish A. Koushik
admin_assigned
2024-11-22T13:17:30.013Z
6065c5161e776d68dd40b18d
/avatars/13f65b99e2b6dbd0be9f88cb5e967175.svg
Girish Koushik
user
mindhunter23
673fe3f3d2dba4a5fb91903c
Zhiyuan Hu
admin_assigned
2024-11-22T13:17:55.835Z
64351475901c5734bcb64248
/avatars/12346d4301c1bfb00ce0ea128a93cc15.svg
Zhiyuan Hu
user
zhiyuanhucs
673fe3f3d2dba4a5fb91903d
Ying Wen
673fe3f3d2dba4a5fb91903e
Jun Wang
Natural Language Reinforcement Learning
Reinforcement Learning (RL) mathematically formulates decision-making with
Markov Decision Process (MDP). With MDPs, researchers have achieved remarkable
breakthroughs across various domains, including games, robotics, and language
models. This paper seeks a new possibility, Natural Language Reinforcement
Learning (NLRL), by extending traditional MDP to natural language-based
representation space. Specifically, NLRL innovatively redefines RL principles,
including task objectives, policy, value function, Bellman equation, and policy
iteration, into their language counterparts. With recent advancements in large
language models (LLMs), NLRL can be practically implemented to achieve RL-like
policy and value improvement by either pure prompting or gradient-based
training. Experiments over Maze, Breakthrough, and Tic-Tac-Toe games
demonstrate the effectiveness, efficiency, and interpretability of the NLRL
framework among diverse use cases. Our code will be released at
https://github.com/waterhorse1/Natural-language-RL.
Natural Language Reinforcement Learning (NLRL) redefines traditional RL concepts in a language-based framework, leveraging large language models to achieve efficient and interpretable policy improvement.
Reinforcement Learning NLRL Markov Decision Process MDP task objectives policy value function Bellman equation policy iteration large language models LLMs gradient-based training