LLM-guided Hierarchical Retrieval
Modern IR systems are increasingly tasked with answering complex,
multi-faceted queries that require deep reasoning rather than simple keyword or
semantic matching. While LLM-based IR has shown great promise, the prevailing
retrieve-then-rerank paradigm inherits the limitations of embedding-based
retrieval; parametric generative approaches are difficult to update with new
information; and long-context methods that place the entire corpus in context
are computationally infeasible for large document collections. To address these
challenges, we introduce LATTICE, a hierarchical retrieval framework that
enables an LLM to reason over and navigate large corpora with logarithmic
search complexity by imposing a semantic tree structure on the corpus. Our
approach consists of two stages: (1) an offline phase that organizes the corpus
into a semantic hierarchy via either a bottom-up agglomerative strategy or a
top-down divisive strategy using multi-level summaries and (2) an online
traversal phase where a search LLM navigates this tree. A central challenge in
such LLM-guided search is that the model's relevance judgments are noisy,
context-dependent, and unaware of the hierarchy, making cross-branch and
cross-level comparisons difficult. To overcome this, we propose a traversal
algorithm that estimates calibrated latent relevance scores from local LLM
outputs and aggregates them into a global path relevance metric. Our
training-free framework achieves state-of-the-art zero-shot performance on the
reasoning-intensive BRIGHT benchmark, demonstrating up to 9% improvement in
Recall@100 and 5% in nDCG@10 over the next best zero-shot baseline.
Furthermore, compared to the fine-tuned SOTA method DIVER-v2, LATTICE attains
comparable results on BRIGHT subsets that use a static corpus for evaluation.
68f19ecf6e0bef323a68fc75
Nilesh Gupta
claimed_verified
2025-10-18T04:49:49.039Z
6527151d5606f146974d60d8
/avatars/00ac8ab005ceadae866dea5471f6aab9.svg
Nilesh Gupta
user
quicktensor
68f19ecf6e0bef323a68fc76
Wei-Cheng Chang
68f19ecf6e0bef323a68fc77
Ngot Bui
68f19ecf6e0bef323a68fc78
Cho-Jui Hsieh
68f19ecf6e0bef323a68fc79
Inderjit S. Dhillon
LLM-guided Hierarchical Retrieval
Modern IR systems are increasingly tasked with answering complex,
multi-faceted queries that require deep reasoning rather than simple keyword or
semantic matching. While LLM-based IR has shown great promise, the prevailing
retrieve-then-rerank paradigm inherits the limitations of embedding-based
retrieval; parametric generative approaches are difficult to update with new
information; and long-context methods that place the entire corpus in context
are computationally infeasible for large document collections. To address these
challenges, we introduce LATTICE, a hierarchical retrieval framework that
enables an LLM to reason over and navigate large corpora with logarithmic
search complexity by imposing a semantic tree structure on the corpus. Our
approach consists of two stages: (1) an offline phase that organizes the corpus
into a semantic hierarchy via either a bottom-up agglomerative strategy or a
top-down divisive strategy using multi-level summaries and (2) an online
traversal phase where a search LLM navigates this tree. A central challenge in
such LLM-guided search is that the model's relevance judgments are noisy,
context-dependent, and unaware of the hierarchy, making cross-branch and
cross-level comparisons difficult. To overcome this, we propose a traversal
algorithm that estimates calibrated latent relevance scores from local LLM
outputs and aggregates them into a global path relevance metric. Our
training-free framework achieves state-of-the-art zero-shot performance on the
reasoning-intensive BRIGHT benchmark, demonstrating up to 9% improvement in
Recall@100 and 5% in nDCG@10 over the next best zero-shot baseline.
Furthermore, compared to the fine-tuned SOTA method DIVER-v2, LATTICE attains
comparable results on BRIGHT subsets that use a static corpus for evaluation.
LATTICE, a hierarchical retrieval framework, enables efficient and accurate reasoning over large document collections using a semantic tree structure and a traversal algorithm that calibrates relevance scores.
LLM-based IR retrieve-then-rerank embedding-based retrieval parametric generative approaches long-context methods hierarchical retrieval framework semantic tree structure offline phase bottom-up agglomerative strategy top-down divisive strategy multi-level summaries online traversal phase search LLM relevance judgments latent relevance scores global path relevance metric zero-shot performance BRIGHT benchmark Recall@100 nDCG@10 DIVER-v2