Mem-α: Learning Memory Construction via Reinforcement Learning
Large language model (LLM) agents are constrained by limited context windows,
necessitating external memory systems for long-term information understanding.
Current memory-augmented agents typically depend on pre-defined instructions
and tools for memory updates. However, language models may lack the ability to
determine which information to store, how to structure it, and when to update
it, especially as memory systems become more complex. This results in
suboptimal memory construction and information loss. To this end, we propose
Mem-alpha, a reinforcement learning framework that trains agents to effectively
manage complex memory systems through interaction and feedback. We also
construct a specialized training dataset spanning diverse multi-turn
interaction patterns paired with comprehensive evaluation questions designed to
teach effective memory management. During training, agents process sequential
information chunks, learn to extract and store relevant content, then update
the memory system. The reward signal derives from downstream question-answering
accuracy over the full interaction history, directly optimizing for memory
construction. To illustrate the effectiveness of our training framework, we
design a memory architecture comprising core, episodic, and semantic
components, equipped with multiple tools for memory operations. Empirical
evaluation demonstrates that Mem-alpha achieves significant improvements over
existing memory-augmented agent baselines. Despite being trained exclusively on
instances with a maximum length of 30k tokens, our agents exhibit remarkable
generalization to sequences exceeding 400k tokens, over 13x the training
length, highlighting the robustness of Mem-alpha.
68dcc1a04159d1f2418f9c28
Yu Wang
68dcc1a04159d1f2418f9c29
Ryuichi Takanobu
68dcc1a04159d1f2418f9c2a
Zhiqi Liang
claimed_verified
2025-10-01T10:20:49.318Z
68c8ff3df0714530f2d75b22
/avatars/984b32090b9026dc31d54b93e9ac6a9e.svg
Zhiqi Liang
user
zkadelzq
68dcc1a04159d1f2418f9c2b
Yuzhen Mao
68dcc1a04159d1f2418f9c2c
Yuanzhe Hu
claimed_verified
2025-10-01T10:20:51.430Z
686647f3e2cca4bc45818704
/avatars/1efeff78892223b6e5592d4ef994f7c3.svg
YUANZHE HU
user
ai-hyz
68dcc1a04159d1f2418f9c2d
Julian McAuley
68dcc1a04159d1f2418f9c2e
Xiaojian Wu
Mem-α: Learning Memory Construction via Reinforcement Learning
Large language model (LLM) agents are constrained by limited context windows,
necessitating external memory systems for long-term information understanding.
Current memory-augmented agents typically depend on pre-defined instructions
and tools for memory updates. However, language models may lack the ability to
determine which information to store, how to structure it, and when to update
it, especially as memory systems become more complex. This results in
suboptimal memory construction and information loss. To this end, we propose
Mem-alpha, a reinforcement learning framework that trains agents to effectively
manage complex memory systems through interaction and feedback. We also
construct a specialized training dataset spanning diverse multi-turn
interaction patterns paired with comprehensive evaluation questions designed to
teach effective memory management. During training, agents process sequential
information chunks, learn to extract and store relevant content, then update
the memory system. The reward signal derives from downstream question-answering
accuracy over the full interaction history, directly optimizing for memory
construction. To illustrate the effectiveness of our training framework, we
design a memory architecture comprising core, episodic, and semantic
components, equipped with multiple tools for memory operations. Empirical
evaluation demonstrates that Mem-alpha achieves significant improvements over
existing memory-augmented agent baselines. Despite being trained exclusively on
instances with a maximum length of 30k tokens, our agents exhibit remarkable
generalization to sequences exceeding 400k tokens, over 13x the training
length, highlighting the robustness of Mem-alpha.
Mem-alpha, a reinforcement learning framework, enhances memory management in large language models through interaction and feedback, improving performance and generalization in long-term information understanding.
reinforcement learning memory-augmented agents memory management interaction patterns episodic memory semantic memory memory operations question-answering accuracy generalization