A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
Current Large Language Models (LLMs) are not only limited to some maximum
context length, but also are not able to robustly consume long inputs. To
address these limitations, we propose ReadAgent, an LLM agent system that
increases effective context length up to 20x in our experiments. Inspired by
how humans interactively read long documents, we implement ReadAgent as a
simple prompting system that uses the advanced language capabilities of LLMs to
(1) decide what content to store together in a memory episode, (2) compress
those memory episodes into short episodic memories called gist memories, and
(3) take actions to look up passages in the original text if ReadAgent needs to
remind itself of relevant details to complete a task. We evaluate ReadAgent
against baselines using retrieval methods, using the original long contexts,
and using the gist memories. These evaluations are performed on three
long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum.
ReadAgent outperforms the baselines on all three tasks while extending the
effective context window by 3-20x.
65ced76da0f4d749b2e6f9cd
Kuang-Huei Lee
admin_assigned
2025-01-20T13:17:48.082Z
669fbf44200c01b737751dc5
/avatars/2023c01b2a8cc1625cafcd0b625871dc.svg
Kuang-Huei Lee
user
khlee112
65ced76da0f4d749b2e6f9ce
Xinyun Chen
admin_assigned
2024-02-16T11:09:55.834Z
64d0268001931c60161e026a
/avatars/0ea48c47b0270b449fd6b97b495e64a6.svg
Xinyun Chen
user
xinyunchen
65ced76da0f4d749b2e6f9cf
Hiroki Furuta
admin_assigned
2024-02-16T11:10:02.407Z
646c39fc3e2a7b06594bac6c
/avatars/8925fef2b6e7105a0f0a4bbf160cfda3.svg
Hiroki Furuta
user
frt03
65ced76da0f4d749b2e6f9d0
John Canny
65ced76da0f4d749b2e6f9d1
Ian Fischer
admin_assigned
2024-02-16T11:10:15.054Z
64ef4f9866f36326b3ec5b8c
/avatars/9eea179e0797f952664f33e1aef21e88.svg
Ian Fischer
user
Ianfischer
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
Current Large Language Models (LLMs) are not only limited to some maximum
context length, but also are not able to robustly consume long inputs. To
address these limitations, we propose ReadAgent, an LLM agent system that
increases effective context length up to 20x in our experiments. Inspired by
how humans interactively read long documents, we implement ReadAgent as a
simple prompting system that uses the advanced language capabilities of LLMs to
(1) decide what content to store together in a memory episode, (2) compress
those memory episodes into short episodic memories called gist memories, and
(3) take actions to look up passages in the original text if ReadAgent needs to
remind itself of relevant details to complete a task. We evaluate ReadAgent
against baselines using retrieval methods, using the original long contexts,
and using the gist memories. These evaluations are performed on three
long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum.
ReadAgent outperforms the baselines on all three tasks while extending the
effective context window by 3-20x.
ReadAgent, an LLM-based prompting system, enhances effective context length up to 20x through memory episodes and gist memories, outperforming baselines on long-document reading comprehension tasks.
ReadAgent large language models (LLMs) memory episodes gist memories long-document reading comprehension QuALITY NarrativeQA QMSum