Learning on the Job: An Experience-Driven Self-Evolving Agent for
  Long-Horizon Tasks
Large Language Models have demonstrated remarkable capabilities across
diverse domains, yet significant challenges persist when deploying them as AI
agents for real-world long-horizon tasks. Existing LLM agents suffer from a
critical limitation: they are test-time static and cannot learn from
experience, lacking the ability to accumulate knowledge and continuously
improve on the job. To address this challenge, we propose MUSE, a novel agent
framework that introduces an experience-driven, self-evolving system centered
around a hierarchical Memory Module. MUSE organizes diverse levels of
experience and leverages them to plan and execute long-horizon tasks across
multiple applications. After each sub-task execution, the agent autonomously
reflects on its trajectory, converting the raw trajectory into structured
experience and integrating it back into the Memory Module. This mechanism
enables the agent to evolve beyond its static pretrained parameters, fostering
continuous learning and self-evolution. We evaluate MUSE on the long-horizon
productivity benchmark TAC. It achieves new SOTA performance by a significant
margin using only a lightweight Gemini-2.5 Flash model. Sufficient Experiments
demonstrate that as the agent autonomously accumulates experience, it exhibits
increasingly superior task completion capabilities, as well as robust
continuous learning and self-evolution capabilities. Moreover, the accumulated
experience from MUSE exhibits strong generalization properties, enabling
zero-shot improvement on new tasks. MUSE establishes a new paradigm for AI
agents capable of real-world productivity task automation.
68e872a695e8e6771df389bf
Cheng Yang
claimed_verified
2025-10-10T08:56:17.687Z
64b289681917256b41140bf7
/avatars/22309802f680ad38310d53d4b008ab21.svg
YcHades
user
YcHades
68e872a695e8e6771df389c0
Xuemeng Yang
claimed_verified
2025-10-10T08:57:50.348Z
656882da5025f8e01b424eda
/avatars/7fda244634c90b4647b49f83fdb25a9b.svg
yxm
user
jokester-yxm
68e872a695e8e6771df389c1
Licheng Wen
claimed_verified
2025-10-10T08:57:56.780Z
64a7c43ae940d769194055df
/avatars/441ccadd62e039fb8cb112f138ed917d.svg
Licheng Wen
user
Wayne-lc
68e872a695e8e6771df389c2
Daocheng Fu
claimed_verified
2025-10-10T04:32:07.385Z
64673506b990713c503079bc
/avatars/17396c835f1eeae00ec2972b6a24a2dd.svg
Fu
user
fudaocheng
68e872a695e8e6771df389c3
Jianbiao Mei
claimed_verified
2025-10-10T08:55:40.955Z
65827953d73d6402f7ae332d
/avatars/d57eae62171ece7a237eaf4ea938a31c.svg
Mei
user
Jianbiao
68e872a695e8e6771df389c4
Rong Wu
68e872a695e8e6771df389c5
Pinlong Cai
68e872a695e8e6771df389c6
Yufan Shen
claimed_verified
2025-10-17T04:14:32.704Z
68ef824d0cac013e4a83f7c1
/avatars/4737edbadd9b4b78968884de30664ffd.svg
shenyfzju
user
shenyfzju
68e872a695e8e6771df389c7
Nianchen Deng
68e872a695e8e6771df389c8
Botian Shi
68e872a695e8e6771df389c9
Yu Qiao
68e872a695e8e6771df389ca
Haifeng Li
Learning on the Job: An Experience-Driven Self-Evolving Agent for
  Long-Horizon Tasks
Large Language Models have demonstrated remarkable capabilities across
diverse domains, yet significant challenges persist when deploying them as AI
agents for real-world long-horizon tasks. Existing LLM agents suffer from a
critical limitation: they are test-time static and cannot learn from
experience, lacking the ability to accumulate knowledge and continuously
improve on the job. To address this challenge, we propose MUSE, a novel agent
framework that introduces an experience-driven, self-evolving system centered
around a hierarchical Memory Module. MUSE organizes diverse levels of
experience and leverages them to plan and execute long-horizon tasks across
multiple applications. After each sub-task execution, the agent autonomously
reflects on its trajectory, converting the raw trajectory into structured
experience and integrating it back into the Memory Module. This mechanism
enables the agent to evolve beyond its static pretrained parameters, fostering
continuous learning and self-evolution. We evaluate MUSE on the long-horizon
productivity benchmark TAC. It achieves new SOTA performance by a significant
margin using only a lightweight Gemini-2.5 Flash model. Sufficient Experiments
demonstrate that as the agent autonomously accumulates experience, it exhibits
increasingly superior task completion capabilities, as well as robust
continuous learning and self-evolution capabilities. Moreover, the accumulated
experience from MUSE exhibits strong generalization properties, enabling
zero-shot improvement on new tasks. MUSE establishes a new paradigm for AI
agents capable of real-world productivity task automation.
MUSE, a novel agent framework with a hierarchical Memory Module, enables continuous learning and self-evolution, achieving state-of-the-art performance on long-horizon productivity tasks using a lightweight model.
Large Language Models LLM agents experience-driven self-evolving system hierarchical Memory Module long-horizon tasks trajectory reflection structured experience continuous learning self-evolution TAC benchmark Gemini-2.5 Flash model zero-shot improvement