MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge
  Discovery
Retrieval-Augmented Generation (RAG) leverages retrieval tools to access
external databases, thereby enhancing the generation quality of large language
models (LLMs) through optimized context. However, the existing retrieval
methods are constrained inherently, as they can only perform relevance matching
between explicitly stated queries and well-formed knowledge, but unable to
handle tasks involving ambiguous information needs or unstructured knowledge.
Consequently, existing RAG systems are primarily effective for straightforward
question-answering tasks. In this work, we propose MemoRAG, a novel
retrieval-augmented generation paradigm empowered by long-term memory. MemoRAG
adopts a dual-system architecture. On the one hand, it employs a light
but long-range LLM to form the global memory of database. Once a task is
presented, it generates draft answers, cluing the retrieval tools to locate
useful information within the database. On the other hand, it leverages an
expensive but expressive LLM, which generates the ultimate answer
based on the retrieved information. Building on this general framework, we
further optimize MemoRAG's performance by enhancing its cluing mechanism and
memorization capacity. In our experiment, MemoRAG achieves superior performance
across a variety of evaluation tasks, including both complex ones where
conventional RAG fails and straightforward ones where RAG is commonly applied.
66e000965511a2f2f7325260
Hongjin Qian
66e000965511a2f2f7325261
Peitian Zhang
admin_assigned
2024-09-10T08:38:21.338Z
603fb3ebabc4bc39998ff765
/avatars/1a20dec6a1e48017f9a650ac2e510b4e.svg
NaN
user
namespace-Pt
66e000965511a2f2f7325262
Zheng Liu
claimed_verified
2024-09-10T12:26:00.728Z
64a38c590111d5ff6c3d5f2b
/avatars/ef13dc7ce243819bc0da9b04e778b432.svg
zhengliu
user
zl101
66e000965511a2f2f7325263
Kelong Mao
claimed_verified
2024-12-30T19:38:51.926Z
649a65605c74a2125c22bbc1
/avatars/e7435d3aeeb59acc6f6f43b48d6982a0.svg
Mao
user
kyriemao
66e000965511a2f2f7325264
Zhicheng Dou
claimed_verified
2024-09-23T06:58:40.243Z
66f0bf59e9d50ec57febf751
/avatars/be97941e60064e5dd806c6fe9db3c537.svg
Zhicheng Dou
user
douzc
MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge
  Discovery
Retrieval-Augmented Generation (RAG) leverages retrieval tools to access
external databases, thereby enhancing the generation quality of large language
models (LLMs) through optimized context. However, the existing retrieval
methods are constrained inherently, as they can only perform relevance matching
between explicitly stated queries and well-formed knowledge, but unable to
handle tasks involving ambiguous information needs or unstructured knowledge.
Consequently, existing RAG systems are primarily effective for straightforward
question-answering tasks. In this work, we propose MemoRAG, a novel
retrieval-augmented generation paradigm empowered by long-term memory. MemoRAG
adopts a dual-system architecture. On the one hand, it employs a light
but long-range LLM to form the global memory of database. Once a task is
presented, it generates draft answers, cluing the retrieval tools to locate
useful information within the database. On the other hand, it leverages an
expensive but expressive LLM, which generates the ultimate answer
based on the retrieved information. Building on this general framework, we
further optimize MemoRAG's performance by enhancing its cluing mechanism and
memorization capacity. In our experiment, MemoRAG achieves superior performance
across a variety of evaluation tasks, including both complex ones where
conventional RAG fails and straightforward ones where RAG is commonly applied.
MemoRAG enhances retrieval-augmented generation by incorporating long-term memory and a dual-system architecture, leading to superior performance across both complex and straightforward tasks.
Retrieval-Augmented Generation (RAG) large language models (LLMs) long-term memory dual-system architecture global memory draft answers retrieval tools ultimate answer cluing mechanism memorization capacity