Don't Just Fine-tune the Agent, Tune the Environment
Large Language Model (LLM) agents show great promise for complex, multi-turn
tool-use tasks, but their development is often hampered by the extreme scarcity
of high-quality training data. Supervised fine-tuning (SFT) on synthetic data
leads to overfitting, whereas standard reinforcement learning (RL) struggles
with a critical cold-start problem and training instability. To address these
challenges, we introduce Environment Tuning, a novel training
paradigm that enables agents to learn complex behaviors directly from problem
instances without relying on pre-collected expert trajectories.
Environment Tuning orchestrates this learning process through a
structured curriculum, actionable environment augmentation that provides
corrective feedback, and fine-grained progress rewards to ensure stable and
efficient exploration. Using only 400 problem instances from Berkeley
Function-Calling Leaderboard (BFCL) benchmark, our method not only achieves
competitive in-distribution performance against strong baselines but also
demonstrates superior out-of-distribution generalization, overcoming the
performance collapse common to SFT-based approaches. Our work presents a
paradigm shift from supervised fine-tuning on static trajectories to dynamic,
environment-based exploration, paving the way for training more robust and
data-efficient agents.
68edd8cede1fee572713a995
Siyuan Lu
claimed_verified
2025-10-14T07:29:30.092Z
6696954da7fd582ae70db39d
/avatars/92bf42710b206a3164b3ace8090443fa.svg
Siyuan Lu (SII)
user
IcyFish
68edd8cede1fee572713a996
Zechuan Wang
68edd8cede1fee572713a997
Hongxuan Zhang
68edd8cede1fee572713a998
Qintong Wu
claimed_verified
2025-10-14T07:29:27.348Z
68243db9d08d8e01109dd4f4
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/iZKRzoFHewLgAyNVanWf_.png
Qing Wu
user
qingw-dev
68edd8cede1fee572713a999
Leilei Gan
68edd8cede1fee572713a99a
Chenyi Zhuang
claimed_verified
2025-10-14T07:29:25.180Z
64e847ab5ddcace745b8f5b1
/avatars/89525e66ff2900f86f66a11043a298f9.svg
chenyi zhuang
user
chengle
68edd8cede1fee572713a99b
Jinjie Gu
68edd8cede1fee572713a99c
Tao Lin
Don't Just Fine-tune the Agent, Tune the Environment
Large Language Model (LLM) agents show great promise for complex, multi-turn
tool-use tasks, but their development is often hampered by the extreme scarcity
of high-quality training data. Supervised fine-tuning (SFT) on synthetic data
leads to overfitting, whereas standard reinforcement learning (RL) struggles
with a critical cold-start problem and training instability. To address these
challenges, we introduce Environment Tuning, a novel training
paradigm that enables agents to learn complex behaviors directly from problem
instances without relying on pre-collected expert trajectories.
Environment Tuning orchestrates this learning process through a
structured curriculum, actionable environment augmentation that provides
corrective feedback, and fine-grained progress rewards to ensure stable and
efficient exploration. Using only 400 problem instances from Berkeley
Function-Calling Leaderboard (BFCL) benchmark, our method not only achieves
competitive in-distribution performance against strong baselines but also
demonstrates superior out-of-distribution generalization, overcoming the
performance collapse common to SFT-based approaches. Our work presents a
paradigm shift from supervised fine-tuning on static trajectories to dynamic,
environment-based exploration, paving the way for training more robust and
data-efficient agents.
Environment Tuning enables LLM agents to learn complex behaviors from problem instances using a structured curriculum, environment augmentation, and progress rewards, achieving competitive in-distribution performance and superior out-of-distribution generalization.
Environment Tuning structured curriculum environment augmentation progress rewards reinforcement learning supervised fine-tuning LLM agents Berkeley Function-Calling Leaderboard in-distribution performance out-of-distribution generalization