AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs
In this paper, we introduce a novel learning paradigm for adaptive Large
Language Model (LLM) agents that eliminates the need for fine-tuning the
underlying LLMs. Existing approaches are often either rigid, relying on static,
handcrafted reflection workflows, or computationally intensive, requiring
gradient updates of LLM model parameters. In contrast, our method enables
low-cost continual adaptation via memory-based online reinforcement learning.
We formalise this as a Memory-augmented Markov Decision Process (M-MDP),
equipped with a neural case-selection policy to guide action decisions. Past
experiences are stored in an episodic memory, either differentiable or
non-parametric. The policy is continually updated based on environmental
feedback through a memory rewriting mechanism, whereas policy improvement is
achieved through efficient memory reading (retrieval). We instantiate our agent
model in the deep research setting, namely AgentFly, which attains top-1 on
GAIA validation (87.88% Pass@3) and 79.40% on the test set. It reaches
66.6% F1 and 80.4% PM on the DeepResearcher dataset, outperforming the
state-of-the-art training-based method, while case-based memory adds 4.7% to
9.6% absolute points on out-of-distribution tasks. Our approach offers a
scalable and efficient pathway for developing generalist LLM agents capable of
continuous, real-time learning without gradient updates, advancing machine
learning towards open-ended skill acquisition and deep research scenarios. The
code is available at https://github.com/Agent-on-the-Fly/AgentFly.
68abe23f86b21a0e2e358af8
Huichi Zhou
claimed_verified
2025-08-25T08:29:38.720Z
645d7f107c7258d904e82749
/avatars/a4e9d47b281f18616c522c1a8b8ee7e5.svg
HuichiZhou
user
Zhouhc
68abe23f86b21a0e2e358af9
Yihang Chen
claimed_verified
2025-08-25T12:37:49.106Z
66996ea912210698d6fb453b
/avatars/d898f7967d4d0785e0c7a1e94b7a237c.svg
Yihang Chen
user
scyyc9
68abe23f86b21a0e2e358afa
Siyuan Guo
68abe23f86b21a0e2e358afb
Xue Yan
68abe23f86b21a0e2e358afc
Kin Hei Lee
68abe23f86b21a0e2e358afd
Zihan Wang
68abe23f86b21a0e2e358afe
Ka Yiu Lee
claimed_verified
2025-09-02T07:57:16.487Z
65041307324053b21adcf01a
/avatars/0377287cdf55c698c660b8a3c79f43c7.svg
Ka Yiu Lee
user
ycps051031
68abe23f86b21a0e2e358aff
Guchun Zhang
68abe23f86b21a0e2e358b00
Kun Shao
68abe23f86b21a0e2e358b01
Linyi Yang
claimed_verified
2025-08-25T08:18:49.407Z
64895683f534abe18eec264b
/avatars/73cc9e6db6db86793787750776b57c63.svg
Linyi Yang
user
linyiyang2023
68abe23f86b21a0e2e358b02
Jun Wang
AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs
In this paper, we introduce a novel learning paradigm for adaptive Large
Language Model (LLM) agents that eliminates the need for fine-tuning the
underlying LLMs. Existing approaches are often either rigid, relying on static,
handcrafted reflection workflows, or computationally intensive, requiring
gradient updates of LLM model parameters. In contrast, our method enables
low-cost continual adaptation via memory-based online reinforcement learning.
We formalise this as a Memory-augmented Markov Decision Process (M-MDP),
equipped with a neural case-selection policy to guide action decisions. Past
experiences are stored in an episodic memory, either differentiable or
non-parametric. The policy is continually updated based on environmental
feedback through a memory rewriting mechanism, whereas policy improvement is
achieved through efficient memory reading (retrieval). We instantiate our agent
model in the deep research setting, namely AgentFly, which attains top-1 on
GAIA validation (87.88% Pass@3) and 79.40% on the test set. It reaches
66.6% F1 and 80.4% PM on the DeepResearcher dataset, outperforming the
state-of-the-art training-based method, while case-based memory adds 4.7% to
9.6% absolute points on out-of-distribution tasks. Our approach offers a
scalable and efficient pathway for developing generalist LLM agents capable of
continuous, real-time learning without gradient updates, advancing machine
learning towards open-ended skill acquisition and deep research scenarios. The
code is available at https://github.com/Agent-on-the-Fly/AgentFly.
A novel memory-augmented reinforcement learning paradigm enables adaptive LLM agents to continually learn without fine-tuning, using episodic memory and a neural case-selection policy.
Large Language Model (LLM) memory-based online reinforcement learning Memory-augmented Markov Decision Process (M-MDP) neural case-selection policy episodic memory differentiable memory non-parametric memory memory rewriting mechanism memory reading AgentFly GAIA validation DeepResearcher dataset open-ended skill acquisition deep research scenarios