LoongRL:Reinforcement Learning for Advanced Reasoning over Long Contexts
Reasoning over long contexts is essential for large language models. While
reinforcement learning (RL) enhances short-context reasoning by inducing "Aha"
moments in chain-of-thought, the advanced thinking patterns required for
long-context reasoning remain largely unexplored, and high-difficulty RL data
are scarce. In this paper, we introduce LoongRL, a data-driven RL method for
advanced long-context reasoning. Central to LoongRL is KeyChain, a synthesis
approach that transforms short multi-hop QA into high-difficulty long-context
tasks by inserting UUID chains that hide the true question among large
collections of distracting documents. Solving these tasks requires the model to
trace the correct chain step-by-step, identify the true question, retrieve
relevant facts and reason over them to answer correctly. RL training on
KeyChain data induces an emergent plan-retrieve-reason-recheck reasoning
pattern that generalizes far beyond training length. Models trained at 16K
effectively solve 128K tasks without prohibitive full-length RL rollout costs.
On Qwen2.5-7B and 14B, LoongRL substantially improves long-context multi-hop QA
accuracy by +23.5% and +21.1% absolute gains. The resulting LoongRL-14B reaches
a score of 74.2, rivaling much larger frontier models such as o3-mini (74.5)
and DeepSeek-R1 (74.9). It also improves long-context retrieval, passes all
128K needle-in-a-haystack stress tests, and preserves short-context reasoning
capabilities.
68f9884db9b2e4ae0467374e
Siyuan Wang
claimed_verified
2025-10-23T14:51:12.876Z
6495b0b844bc2e9ce6cc849b
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/j6aucl_tefMHwtD-bdUAw.jpeg
Siyuan Wang
user
OldKingMeister
68f9884db9b2e4ae0467374f
Gaokai Zhang
68f9884db9b2e4ae04673750
Li Lyna Zhang
68f9884db9b2e4ae04673751
Ning Shang
68f9884db9b2e4ae04673752
Fan Yang
68f9884db9b2e4ae04673753
Dongyao Chen
68f9884db9b2e4ae04673754
Mao Yang
LoongRL:Reinforcement Learning for Advanced Reasoning over Long Contexts
Reasoning over long contexts is essential for large language models. While
reinforcement learning (RL) enhances short-context reasoning by inducing "Aha"
moments in chain-of-thought, the advanced thinking patterns required for
long-context reasoning remain largely unexplored, and high-difficulty RL data
are scarce. In this paper, we introduce LoongRL, a data-driven RL method for
advanced long-context reasoning. Central to LoongRL is KeyChain, a synthesis
approach that transforms short multi-hop QA into high-difficulty long-context
tasks by inserting UUID chains that hide the true question among large
collections of distracting documents. Solving these tasks requires the model to
trace the correct chain step-by-step, identify the true question, retrieve
relevant facts and reason over them to answer correctly. RL training on
KeyChain data induces an emergent plan-retrieve-reason-recheck reasoning
pattern that generalizes far beyond training length. Models trained at 16K
effectively solve 128K tasks without prohibitive full-length RL rollout costs.
On Qwen2.5-7B and 14B, LoongRL substantially improves long-context multi-hop QA
accuracy by +23.5% and +21.1% absolute gains. The resulting LoongRL-14B reaches
a score of 74.2, rivaling much larger frontier models such as o3-mini (74.5)
and DeepSeek-R1 (74.9). It also improves long-context retrieval, passes all
128K needle-in-a-haystack stress tests, and preserves short-context reasoning
capabilities.
LoongRL, a data-driven reinforcement learning method, enhances long-context reasoning by transforming short multi-hop QA into high-difficulty tasks, improving accuracy and generalization in large language models.
reinforcement learning LoongRL KeyChain multi-hop QA long-context reasoning plan-retrieve-reason-recheck RL rollout Qwen2.5-7B Qwen2.5-14B o3-mini DeepSeek-R1 needle-in-a-haystack stress tests