RAG-Anything: All-in-One RAG Framework
Retrieval-Augmented Generation (RAG) has emerged as a fundamental paradigm
for expanding Large Language Models beyond their static training limitations.
However, a critical misalignment exists between current RAG capabilities and
real-world information environments. Modern knowledge repositories are
inherently multimodal, containing rich combinations of textual content, visual
elements, structured tables, and mathematical expressions. Yet existing RAG
frameworks are limited to textual content, creating fundamental gaps when
processing multimodal documents. We present RAG-Anything, a unified framework
that enables comprehensive knowledge retrieval across all modalities. Our
approach reconceptualizes multimodal content as interconnected knowledge
entities rather than isolated data types. The framework introduces dual-graph
construction to capture both cross-modal relationships and textual semantics
within a unified representation. We develop cross-modal hybrid retrieval that
combines structural knowledge navigation with semantic matching. This enables
effective reasoning over heterogeneous content where relevant evidence spans
multiple modalities. RAG-Anything demonstrates superior performance on
challenging multimodal benchmarks, achieving significant improvements over
state-of-the-art methods. Performance gains become particularly pronounced on
long documents where traditional approaches fail. Our framework establishes a
new paradigm for multimodal knowledge access, eliminating the architectural
fragmentation that constrains current systems. Our framework is open-sourced
at: https://github.com/HKUDS/RAG-Anything.
68efa8045178b1b97206b296
Zirui Guo
68efa8045178b1b97206b297
Xubin Ren
68efa8045178b1b97206b298
Lingrui Xu
68efa8045178b1b97206b299
Jiahao Zhang
68efa8045178b1b97206b29a
Chao Huang
RAG-Anything: All-in-One RAG Framework
Retrieval-Augmented Generation (RAG) has emerged as a fundamental paradigm
for expanding Large Language Models beyond their static training limitations.
However, a critical misalignment exists between current RAG capabilities and
real-world information environments. Modern knowledge repositories are
inherently multimodal, containing rich combinations of textual content, visual
elements, structured tables, and mathematical expressions. Yet existing RAG
frameworks are limited to textual content, creating fundamental gaps when
processing multimodal documents. We present RAG-Anything, a unified framework
that enables comprehensive knowledge retrieval across all modalities. Our
approach reconceptualizes multimodal content as interconnected knowledge
entities rather than isolated data types. The framework introduces dual-graph
construction to capture both cross-modal relationships and textual semantics
within a unified representation. We develop cross-modal hybrid retrieval that
combines structural knowledge navigation with semantic matching. This enables
effective reasoning over heterogeneous content where relevant evidence spans
multiple modalities. RAG-Anything demonstrates superior performance on
challenging multimodal benchmarks, achieving significant improvements over
state-of-the-art methods. Performance gains become particularly pronounced on
long documents where traditional approaches fail. Our framework establishes a
new paradigm for multimodal knowledge access, eliminating the architectural
fragmentation that constrains current systems. Our framework is open-sourced
at: https://github.com/HKUDS/RAG-Anything.
RAG-Anything is a unified framework that enhances multimodal knowledge retrieval by integrating cross-modal relationships and semantic matching, outperforming existing methods on complex benchmarks.
Retrieval-Augmented Generation RAG Large Language Models multimodal textual content visual elements structured tables mathematical expressions dual-graph construction cross-modal hybrid retrieval structural knowledge navigation semantic matching multimodal benchmarks long documents