LightMem: Lightweight and Efficient Memory-Augmented Generation
Despite their remarkable capabilities, Large Language Models (LLMs) struggle
to effectively leverage historical interaction information in dynamic and
complex environments. Memory systems enable LLMs to move beyond stateless
interactions by introducing persistent information storage, retrieval, and
utilization mechanisms. However, existing memory systems often introduce
substantial time and computational overhead. To this end, we introduce a new
memory system called LightMem, which strikes a balance between the performance
and efficiency of memory systems. Inspired by the Atkinson-Shiffrin model of
human memory, LightMem organizes memory into three complementary stages. First,
cognition-inspired sensory memory rapidly filters irrelevant information
through lightweight compression and groups information according to their
topics. Next, topic-aware short-term memory consolidates these topic-based
groups, organizing and summarizing content for more structured access. Finally,
long-term memory with sleep-time update employs an offline procedure that
decouples consolidation from online inference. Experiments on LongMemEval with
GPT and Qwen backbones show that LightMem outperforms strong baselines in
accuracy (up to 10.9% gains) while reducing token usage by up to 117x, API
calls by up to 159x, and runtime by over 12x. The code is available at
https://github.com/zjunlp/LightMem.
68f838947669bcaeecce0c01
Jizhan Fang
68f838947669bcaeecce0c02
Xinle Deng
68f838947669bcaeecce0c03
Haoming Xu
68f838947669bcaeecce0c04
Ziyan Jiang
68f838947669bcaeecce0c05
Yuqi Tang
68f838947669bcaeecce0c06
Ziwen Xu
68f838947669bcaeecce0c07
Shumin Deng
68f838947669bcaeecce0c08
Yunzhi Yao
claimed_verified
2025-10-22T15:44:27.446Z
6122fbe636a4c36a99dbea7b
/avatars/c0cd2c1ef58e315d9adda9d26000f625.svg
Yunzhi Yao
user
cowTodd
68f838947669bcaeecce0c09
Mengru Wang
68f838947669bcaeecce0c0a
Shuofei Qiao
68f838947669bcaeecce0c0b
Huajun Chen
68f838947669bcaeecce0c0c
Ningyu Zhang
claimed_verified
2025-10-22T15:44:29.811Z
620b3bbb0668e435407c8d0a
/avatars/e0fccbb2577d76088e09f054c35cffbc.svg
Ningyu Zhang
user
Ningyu
LightMem: Lightweight and Efficient Memory-Augmented Generation
Despite their remarkable capabilities, Large Language Models (LLMs) struggle
to effectively leverage historical interaction information in dynamic and
complex environments. Memory systems enable LLMs to move beyond stateless
interactions by introducing persistent information storage, retrieval, and
utilization mechanisms. However, existing memory systems often introduce
substantial time and computational overhead. To this end, we introduce a new
memory system called LightMem, which strikes a balance between the performance
and efficiency of memory systems. Inspired by the Atkinson-Shiffrin model of
human memory, LightMem organizes memory into three complementary stages. First,
cognition-inspired sensory memory rapidly filters irrelevant information
through lightweight compression and groups information according to their
topics. Next, topic-aware short-term memory consolidates these topic-based
groups, organizing and summarizing content for more structured access. Finally,
long-term memory with sleep-time update employs an offline procedure that
decouples consolidation from online inference. Experiments on LongMemEval with
GPT and Qwen backbones show that LightMem outperforms strong baselines in
accuracy (up to 10.9% gains) while reducing token usage by up to 117x, API
calls by up to 159x, and runtime by over 12x. The code is available at
https://github.com/zjunlp/LightMem.
LightMem, a memory system inspired by human memory, enhances LLMs by efficiently managing historical interaction information, improving accuracy and reducing computational costs.
Large Language Models LLMs memory systems LightMem Atkinson-Shiffrin model sensory memory topic-aware short-term memory long-term memory sleep-time update LongMemEval GPT Qwen token usage API calls runtime