A Goal Without a Plan Is Just a Wish: Efficient and Effective Global
  Planner Training for Long-Horizon Agent Tasks
Agents based on large language models (LLMs) struggle with brainless
trial-and-error and generating hallucinatory actions due to a lack of global
planning in long-horizon tasks. In this paper, we introduce a plan-and-execute
framework and propose EAGLET, an efficient and effective planner training
method to enhance the executor agent's planning abilities without human effort.
Specifically, we train a plug-and-play global planner through a two-step
process: we first synthesize high-quality plans from an advanced LLM using our
proposed homologous consensus filtering strategy, and apply fine-tuning as a
cold start. Moreover, we further improve the planner with a rule-based
reinforcement learning stage using a novel executor capability gain reward,
ensuring it can handle task instructions of varying difficulty. Experiments on
three long-horizon agent tasks show that executor agents equipped with our
planner outperform existing methods, achieving new state-of-the-art
performance. Meanwhile, EAGLET reduces training costs by 8x compared to
RL-based baselines, and it does not require manual effort or extra training
data, offering an efficient and effective solution.
68ec6e11cd07fb414898c998
Shuzheng Si
68ec6e11cd07fb414898c999
Haozhe Zhao
68ec6e11cd07fb414898c99a
Kangyang Luo
68ec6e11cd07fb414898c99b
Gang Chen
68ec6e11cd07fb414898c99c
Fanchao Qi
68ec6e11cd07fb414898c99d
Minjia Zhang
68ec6e11cd07fb414898c99e
Baobao Chang
68ec6e11cd07fb414898c99f
Maosong Sun
A Goal Without a Plan Is Just a Wish: Efficient and Effective Global
  Planner Training for Long-Horizon Agent Tasks
Agents based on large language models (LLMs) struggle with brainless
trial-and-error and generating hallucinatory actions due to a lack of global
planning in long-horizon tasks. In this paper, we introduce a plan-and-execute
framework and propose EAGLET, an efficient and effective planner training
method to enhance the executor agent's planning abilities without human effort.
Specifically, we train a plug-and-play global planner through a two-step
process: we first synthesize high-quality plans from an advanced LLM using our
proposed homologous consensus filtering strategy, and apply fine-tuning as a
cold start. Moreover, we further improve the planner with a rule-based
reinforcement learning stage using a novel executor capability gain reward,
ensuring it can handle task instructions of varying difficulty. Experiments on
three long-horizon agent tasks show that executor agents equipped with our
planner outperform existing methods, achieving new state-of-the-art
performance. Meanwhile, EAGLET reduces training costs by 8x compared to
RL-based baselines, and it does not require manual effort or extra training
data, offering an efficient and effective solution.
A plan-and-execute framework with EAGLET enhances LLM-based agents' planning abilities, achieving state-of-the-art performance in long-horizon tasks with reduced training costs.
large language models LLMs plan-and-execute framework EAGLET global planner homologous consensus filtering fine-tuning rule-based reinforcement learning executor capability gain reward long-horizon tasks