Memory as Action: Autonomous Context Curation for Long-Horizon Agentic
  Tasks
Large Language Models face challenges in long-horizon agentic tasks as their
constrained memory is easily overwhelmed by distracting or irrelevant context.
Existing working memory methods typically rely on external, heuristic
mechanisms that are decoupled from the agent's core policy. In this work, we
reframe working memory management as a learnable, intrinsic capability. We
propose a novel framework, Memory-as-Action, where an agent actively manages
its working memory by executing explicit editing operations as part of a
unified policy. This formulation allows an agent, trained via reinforcement
learning, to balance memory curation against long-term task objectives under
given resource constraints. However, such memory editing actions break the
standard assumption of a continuously growing prefix in LLM interactions,
leading to what we call trajectory fractures. These non-prefix changes disrupt
the causal continuity required by standard policy gradient methods, making
those methods inapplicable. To address this, we propose a new algorithm,
Dynamic Context Policy Optimization, which enables stable end-to-end
reinforcement learning by segmenting trajectories at memory action points and
applying trajectory-level advantages to the resulting action segments. Our
results demonstrate that jointly optimizing for task reasoning and memory
management in an end-to-end fashion not only reduces overall computational
consumption but also improves task performance, driven by adaptive context
curation strategies tailored to the model's intrinsic capabilities.
68eef8ab486b78128f0e3304
Yuxiang Zhang
claimed_verified
2025-10-15T03:13:02.636Z
645b4a2978730bcc103dfe4d
/avatars/de544de899897fd0a83506ff287123bc.svg
Yuxiang Zhang
user
TokerZ
68eef8ab486b78128f0e3305
Jiangming Shu
68eef8ab486b78128f0e3306
Ye Ma
68eef8ab486b78128f0e3307
Xueyuan Lin
68eef8ab486b78128f0e3308
Shangxi Wu
claimed_verified
2025-10-15T07:07:51.674Z
665ace40ddba0f825f3a4941
/avatars/ed7e209e14b5ae495aa7f4baec980b38.svg
Shangxi Wu
user
KirinNg
68eef8ab486b78128f0e3309
Jitao Sang
Memory as Action: Autonomous Context Curation for Long-Horizon Agentic
  Tasks
Large Language Models face challenges in long-horizon agentic tasks as their
constrained memory is easily overwhelmed by distracting or irrelevant context.
Existing working memory methods typically rely on external, heuristic
mechanisms that are decoupled from the agent's core policy. In this work, we
reframe working memory management as a learnable, intrinsic capability. We
propose a novel framework, Memory-as-Action, where an agent actively manages
its working memory by executing explicit editing operations as part of a
unified policy. This formulation allows an agent, trained via reinforcement
learning, to balance memory curation against long-term task objectives under
given resource constraints. However, such memory editing actions break the
standard assumption of a continuously growing prefix in LLM interactions,
leading to what we call trajectory fractures. These non-prefix changes disrupt
the causal continuity required by standard policy gradient methods, making
those methods inapplicable. To address this, we propose a new algorithm,
Dynamic Context Policy Optimization, which enables stable end-to-end
reinforcement learning by segmenting trajectories at memory action points and
applying trajectory-level advantages to the resulting action segments. Our
results demonstrate that jointly optimizing for task reasoning and memory
management in an end-to-end fashion not only reduces overall computational
consumption but also improves task performance, driven by adaptive context
curation strategies tailored to the model's intrinsic capabilities.