MoM: Mixtures of Scenario-Aware Document Memories for
  Retrieval-Augmented Generation Systems
The traditional RAG paradigm, which typically engages in the comprehension of
relevant text chunks in response to received queries, inherently restricts both
the depth of knowledge internalization and reasoning capabilities. To address
this limitation, our research transforms the text processing in RAG from
passive chunking to proactive understanding, defining this process as document
memory extraction with the objective of simulating human cognitive processes
during reading. Building upon this, we propose the Mixtures of scenario-aware
document Memories (MoM) framework, engineered to efficiently handle documents
from multiple domains and train small language models (SLMs) to acquire the
ability to proactively explore and construct document memories. The MoM
initially instructs large language models (LLMs) to simulate domain experts in
generating document logical outlines, thereby directing structured chunking and
core content extraction. It employs a multi-path sampling and multi-perspective
evaluation mechanism, specifically designing comprehensive metrics that
represent chunk clarity and extraction completeness to select the optimal
document memories. Additionally, to infuse deeper human-like reading abilities
during the training of SLMs, we incorporate a reverse reasoning strategy, which
deduces refined expert thinking paths from high-quality outcomes. Finally,
leveraging diverse forms of content generated by MoM, we develop a three-layer
document memory retrieval mechanism, which is grounded in our theoretical proof
from the perspective of probabilistic modeling. Extensive experimental results
across three distinct domains demonstrate that the MoM framework not only
resolves text chunking challenges in existing RAG systems, providing LLMs with
semantically complete document memories, but also paves the way for SLMs to
achieve human-centric intelligent text processing.
68f1a46e6e0bef323a68fd12
Jihao Zhao
claimed_verified
2025-10-17T04:10:33.450Z
658e85bb5b7553ca5c29ba89
https://cdn-avatars.huggingface.co/v1/production/uploads/658e85bb5b7553ca5c29ba89/KK6UpS9agtrxevvBoup5N.jpeg
Jihao Zhao
user
Robot2050
68f1a46e6e0bef323a68fd13
Zhiyuan Ji
68f1a46e6e0bef323a68fd14
Simin Niu
68f1a46e6e0bef323a68fd15
Hanyu Wang
68f1a46e6e0bef323a68fd16
Feiyu Xiong
68f1a46e6e0bef323a68fd17
Zhiyu Li
MoM: Mixtures of Scenario-Aware Document Memories for
  Retrieval-Augmented Generation Systems
The traditional RAG paradigm, which typically engages in the comprehension of
relevant text chunks in response to received queries, inherently restricts both
the depth of knowledge internalization and reasoning capabilities. To address
this limitation, our research transforms the text processing in RAG from
passive chunking to proactive understanding, defining this process as document
memory extraction with the objective of simulating human cognitive processes
during reading. Building upon this, we propose the Mixtures of scenario-aware
document Memories (MoM) framework, engineered to efficiently handle documents
from multiple domains and train small language models (SLMs) to acquire the
ability to proactively explore and construct document memories. The MoM
initially instructs large language models (LLMs) to simulate domain experts in
generating document logical outlines, thereby directing structured chunking and
core content extraction. It employs a multi-path sampling and multi-perspective
evaluation mechanism, specifically designing comprehensive metrics that
represent chunk clarity and extraction completeness to select the optimal
document memories. Additionally, to infuse deeper human-like reading abilities
during the training of SLMs, we incorporate a reverse reasoning strategy, which
deduces refined expert thinking paths from high-quality outcomes. Finally,
leveraging diverse forms of content generated by MoM, we develop a three-layer
document memory retrieval mechanism, which is grounded in our theoretical proof
from the perspective of probabilistic modeling. Extensive experimental results
across three distinct domains demonstrate that the MoM framework not only
resolves text chunking challenges in existing RAG systems, providing LLMs with
semantically complete document memories, but also paves the way for SLMs to
achieve human-centric intelligent text processing.
The MoM framework enhances RAG by transforming text processing from passive chunking to proactive understanding, enabling LLMs to generate structured document memories and SLMs to develop human-like reading abilities.
document memory extraction Mixtures of scenario-aware document Memories (MoM) small language models (SLMs) large language models (LLMs) document logical outlines multi-path sampling multi-perspective evaluation reverse reasoning strategy three-layer document memory retrieval mechanism probabilistic modeling