MindSearch: Mimicking Human Minds Elicits Deep AI Searcher
Information seeking and integration is a complex cognitive task that consumes
enormous time and effort. Inspired by the remarkable progress of Large Language
Models, recent works attempt to solve this task by combining LLMs and search
engines. However, these methods still obtain unsatisfying performance due to
three challenges: (1) complex requests often cannot be accurately and
completely retrieved by the search engine once (2) corresponding information to
be integrated is spread over multiple web pages along with massive noise, and
(3) a large number of web pages with long contents may quickly exceed the
maximum context length of LLMs. Inspired by the cognitive process when humans
solve these problems, we introduce MindSearch to mimic the human minds in web
information seeking and integration, which can be instantiated by a simple yet
effective LLM-based multi-agent framework. The WebPlanner models the human mind
of multi-step information seeking as a dynamic graph construction process: it
decomposes the user query into atomic sub-questions as nodes in the graph and
progressively extends the graph based on the search result from WebSearcher.
Tasked with each sub-question, WebSearcher performs hierarchical information
retrieval with search engines and collects valuable information for WebPlanner.
The multi-agent design of MindSearch enables the whole framework to seek and
integrate information parallelly from larger-scale (e.g., more than 300) web
pages in 3 minutes, which is worth 3 hours of human effort. MindSearch
demonstrates significant improvement in the response quality in terms of depth
and breadth, on both close-set and open-set QA problems. Besides, responses
from MindSearch based on InternLM2.5-7B are preferable by humans to ChatGPT-Web
and Perplexity.ai applications, which implies that MindSearch can already
deliver a competitive solution to the proprietary AI search engine.
66a85f2f5bd5c24dec0ed484
Zehui Chen
admin_assigned
2024-07-30T08:58:42.500Z
64892d31cbda0d1cdb956897
/avatars/3cdafe03a8295124636347d15a099aaf.svg
Zehui Chen
user
lovesnowbest
66a85f2f5bd5c24dec0ed485
Kuikun Liu
claimed_verified
2024-08-02T07:34:59.738Z
63fd691794cc8f815d50c112
/avatars/87305d1cbfcc717e910ccdfaf0568f80.svg
liu
user
Harold-lkk
66a85f2f5bd5c24dec0ed486
Qiuchen Wang
claimed_verified
2025-02-28T12:18:06.779Z
657429d833e5a4bf5b278615
/avatars/ed7e28c1b9a7bed1cad864c992cdcc69.svg
QiuchenWang
user
autumncc
66a85f2f5bd5c24dec0ed487
Jiangning Liu
admin_assigned
2024-08-06T07:45:32.542Z
65a7646581cc3017643a217d
/avatars/ffc62e6f774e7d76b6a38aff432e01e4.svg
Jiangning Liu
user
liujiangning
66a85f2f5bd5c24dec0ed488
Wenwei Zhang
admin_assigned
2024-08-06T07:45:38.673Z
64e8505321540e1da3226b54
/avatars/18958b8406d1ce492b54c1c839f18c54.svg
Wenwei Zhang
user
ZwwWayne
66a85f2f5bd5c24dec0ed489
Kai Chen
66a85f2f5bd5c24dec0ed48a
Feng Zhao
MindSearch: Mimicking Human Minds Elicits Deep AI Searcher
Information seeking and integration is a complex cognitive task that consumes
enormous time and effort. Inspired by the remarkable progress of Large Language
Models, recent works attempt to solve this task by combining LLMs and search
engines. However, these methods still obtain unsatisfying performance due to
three challenges: (1) complex requests often cannot be accurately and
completely retrieved by the search engine once (2) corresponding information to
be integrated is spread over multiple web pages along with massive noise, and
(3) a large number of web pages with long contents may quickly exceed the
maximum context length of LLMs. Inspired by the cognitive process when humans
solve these problems, we introduce MindSearch to mimic the human minds in web
information seeking and integration, which can be instantiated by a simple yet
effective LLM-based multi-agent framework. The WebPlanner models the human mind
of multi-step information seeking as a dynamic graph construction process: it
decomposes the user query into atomic sub-questions as nodes in the graph and
progressively extends the graph based on the search result from WebSearcher.
Tasked with each sub-question, WebSearcher performs hierarchical information
retrieval with search engines and collects valuable information for WebPlanner.
The multi-agent design of MindSearch enables the whole framework to seek and
integrate information parallelly from larger-scale (e.g., more than 300) web
pages in 3 minutes, which is worth 3 hours of human effort. MindSearch
demonstrates significant improvement in the response quality in terms of depth
and breadth, on both close-set and open-set QA problems. Besides, responses
from MindSearch based on InternLM2.5-7B are preferable by humans to ChatGPT-Web
and Perplexity.ai applications, which implies that MindSearch can already
deliver a competitive solution to the proprietary AI search engine.
MindSearch, an LLM-based multi-agent framework, improves web information seeking and integration through parallel processing and hierarchical retrieval, achieving better performance than existing solutions.
Large Language Models LLMs multi-agent framework dynamic graph construction hierarchical information retrieval WebPlanner WebSearcher close-set QA problems open-set QA problems InternLM2.5-7B ChatGPT-Web Perplexity.ai