Feedback-Driven Tool-Use Improvements in Large Language Models via
  Automated Build Environments
Effective tool use is essential for large language models (LLMs) to interact
meaningfully with their environment. However, progress is limited by the lack
of efficient reinforcement learning (RL) frameworks specifically designed for
tool use, due to challenges in constructing stable training environments and
designing verifiable reward mechanisms. To address this, we propose an
automated environment construction pipeline, incorporating scenario
decomposition, document generation, function integration, complexity scaling,
and localized deployment. This enables the creation of high-quality training
environments that provide detailed and measurable feedback without relying on
external tools. Additionally, we introduce a verifiable reward mechanism that
evaluates both the precision of tool use and the completeness of task
execution. When combined with trajectory data collected from the constructed
environments, this mechanism integrates seamlessly with standard RL algorithms
to facilitate feedback-driven model training. Experiments on LLMs of varying
scales demonstrate that our approach significantly enhances the models'
tool-use performance without degrading their general capabilities, regardless
of inference modes or training algorithms. Our analysis suggests that these
gains result from improved context understanding and reasoning, driven by
updates to the lower-layer MLP parameters in models.
689c08b8fab6fdd2e52ac871
Junjie Ye
claimed_verified
2025-08-13T07:12:31.844Z
66384be673c2c55f2ded89fa
/avatars/1d8721074f0f51fab405f81474f2035f.svg
Junjie Ye
user
Junjie-Ye
689c08b8fab6fdd2e52ac872
Changhao Jiang
689c08b8fab6fdd2e52ac873
Zhengyin Du
689c08b8fab6fdd2e52ac874
Yufei Xu
689c08b8fab6fdd2e52ac875
Xuesong Yao
689c08b8fab6fdd2e52ac876
Zhiheng Xi
689c08b8fab6fdd2e52ac877
Xiaoran Fan
689c08b8fab6fdd2e52ac878
Qi Zhang
689c08b8fab6fdd2e52ac879
Xuanjing Huang
689c08b8fab6fdd2e52ac87a
Jiecao Chen
Feedback-Driven Tool-Use Improvements in Large Language Models via
  Automated Build Environments
Effective tool use is essential for large language models (LLMs) to interact
meaningfully with their environment. However, progress is limited by the lack
of efficient reinforcement learning (RL) frameworks specifically designed for
tool use, due to challenges in constructing stable training environments and
designing verifiable reward mechanisms. To address this, we propose an
automated environment construction pipeline, incorporating scenario
decomposition, document generation, function integration, complexity scaling,
and localized deployment. This enables the creation of high-quality training
environments that provide detailed and measurable feedback without relying on
external tools. Additionally, we introduce a verifiable reward mechanism that
evaluates both the precision of tool use and the completeness of task
execution. When combined with trajectory data collected from the constructed
environments, this mechanism integrates seamlessly with standard RL algorithms
to facilitate feedback-driven model training. Experiments on LLMs of varying
scales demonstrate that our approach significantly enhances the models'
tool-use performance without degrading their general capabilities, regardless
of inference modes or training algorithms. Our analysis suggests that these
gains result from improved context understanding and reasoning, driven by
updates to the lower-layer MLP parameters in models.
An automated pipeline for constructing training environments and a verifiable reward mechanism enhance large language models' tool-use performance without compromising general capabilities.
reinforcement learning scenario decomposition document generation function integration complexity scaling localized deployment verifiable reward mechanism trajectory data standard RL algorithms context understanding reasoning MLP parameters