Self-Taught Agentic Long Context Understanding
Answering complex, long-context questions remains a major challenge for large
language models (LLMs) as it requires effective question clarifications and
context retrieval. We propose Agentic Long-Context Understanding (AgenticLU), a
framework designed to enhance an LLM's understanding of such queries by
integrating targeted self-clarification with contextual grounding within an
agentic workflow. At the core of AgenticLU is Chain-of-Clarifications (CoC),
where models refine their understanding through self-generated clarification
questions and corresponding contextual groundings. By scaling inference as a
tree search where each node represents a CoC step, we achieve 97.8% answer
recall on NarrativeQA with a search depth of up to three and a branching factor
of eight. To amortize the high cost of this search process to training, we
leverage the preference pairs for each step obtained by the CoC workflow and
perform two-stage model finetuning: (1) supervised finetuning to learn
effective decomposition strategies, and (2) direct preference optimization to
enhance reasoning quality. This enables AgenticLU models to generate
clarifications and retrieve relevant context effectively and efficiently in a
single inference pass. Extensive experiments across seven long-context tasks
demonstrate that AgenticLU significantly outperforms state-of-the-art prompting
methods and specialized long-context LLMs, achieving robust multi-hop reasoning
while sustaining consistent performance as context length grows.
67bd1ae085a048f74fd1b8ec
Yufan Zhuang
claimed_verified
2025-02-26T08:38:05.314Z
6438ccbb3b46237de3d052e8
/avatars/baa624d417b0b905e82127dc66346478.svg
Yufan Zhuang
user
yzhuang
67bd1ae085a048f74fd1b8ed
Xiaodong Yu
67bd1ae085a048f74fd1b8ee
Jialian Wu
67bd1ae085a048f74fd1b8ef
Ximeng Sun
67bd1ae085a048f74fd1b8f0
Ze Wang
67bd1ae085a048f74fd1b8f1
Jiang Liu
67bd1ae085a048f74fd1b8f2
Yusheng Su
67bd1ae085a048f74fd1b8f3
Jingbo Shang
67bd1ae085a048f74fd1b8f4
Zicheng Liu
67bd1ae085a048f74fd1b8f5
Emad Barsoum
Self-Taught Agentic Long Context Understanding
Answering complex, long-context questions remains a major challenge for large
language models (LLMs) as it requires effective question clarifications and
context retrieval. We propose Agentic Long-Context Understanding (AgenticLU), a
framework designed to enhance an LLM's understanding of such queries by
integrating targeted self-clarification with contextual grounding within an
agentic workflow. At the core of AgenticLU is Chain-of-Clarifications (CoC),
where models refine their understanding through self-generated clarification
questions and corresponding contextual groundings. By scaling inference as a
tree search where each node represents a CoC step, we achieve 97.8% answer
recall on NarrativeQA with a search depth of up to three and a branching factor
of eight. To amortize the high cost of this search process to training, we
leverage the preference pairs for each step obtained by the CoC workflow and
perform two-stage model finetuning: (1) supervised finetuning to learn
effective decomposition strategies, and (2) direct preference optimization to
enhance reasoning quality. This enables AgenticLU models to generate
clarifications and retrieve relevant context effectively and efficiently in a
single inference pass. Extensive experiments across seven long-context tasks
demonstrate that AgenticLU significantly outperforms state-of-the-art prompting
methods and specialized long-context LLMs, achieving robust multi-hop reasoning
while sustaining consistent performance as context length grows.
AgenticLU enhances LLMs by integrating self-clarification and contextual grounding, improving long-context understanding and multi-hop reasoning through a chain-of-clarifications workflow.
AgenticLU Chain-of-Clarifications CoC self-clarification contextual grounding tree search NarrativeQA answer recall preference pairs supervised finetuning direct preference optimization long-context tasks multi-hop reasoning