A^2Search: Ambiguity-Aware Question Answering with Reinforcement
  Learning
Recent advances in Large Language Models (LLMs) and Reinforcement Learning
(RL) have led to strong performance in open-domain question answering (QA).
However, existing models still struggle with questions that admit multiple
valid answers. Standard QA benchmarks, which typically assume a single gold
answer, overlook this reality and thus produce inappropriate training signals.
Existing attempts to handle ambiguity often rely on costly manual annotation,
which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue.
In this paper, we present A^2Search, an annotation-free, end-to-end training
framework to recognize and handle ambiguity. At its core is an automated
pipeline that detects ambiguous questions and gathers alternative answers via
trajectory sampling and evidence verification. The model is then optimized with
RL using a carefully designed AnsF1 reward, which naturally
accommodates multiple answers. Experiments on eight open-domain QA benchmarks
demonstrate that A^2Search achieves new state-of-the-art performance. With
only a single rollout, A^2Search-7B yields an average AnsF1@1
score of 48.4% across four multi-hop benchmarks, outperforming all strong
baselines, including the substantially larger ReSearch-32B (46.2%).
Extensive analyses further show that A^2Search resolves ambiguity and
generalizes across benchmarks, highlighting that embracing ambiguity is
essential for building more reliable QA systems. Our code, data, and model
weights can be found at https://github.com/zfj1998/A2Search
68e871e695e8e6771df3897b
Fengji Zhang
68e871e695e8e6771df3897c
Xinyao Niu
68e871e695e8e6771df3897d
Chengyang Ying
68e871e695e8e6771df3897e
Guancheng Lin
68e871e695e8e6771df3897f
Zhongkai Hao
68e871e695e8e6771df38980
Zhou Fan
68e871e695e8e6771df38981
Chengen Huang
68e871e695e8e6771df38982
Jacky Keung
68e871e695e8e6771df38983
Bei Chen
68e871e695e8e6771df38984
Junyang Lin
A^2Search: Ambiguity-Aware Question Answering with Reinforcement
  Learning
Recent advances in Large Language Models (LLMs) and Reinforcement Learning
(RL) have led to strong performance in open-domain question answering (QA).
However, existing models still struggle with questions that admit multiple
valid answers. Standard QA benchmarks, which typically assume a single gold
answer, overlook this reality and thus produce inappropriate training signals.
Existing attempts to handle ambiguity often rely on costly manual annotation,
which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue.
In this paper, we present A^2Search, an annotation-free, end-to-end training
framework to recognize and handle ambiguity. At its core is an automated
pipeline that detects ambiguous questions and gathers alternative answers via
trajectory sampling and evidence verification. The model is then optimized with
RL using a carefully designed AnsF1 reward, which naturally
accommodates multiple answers. Experiments on eight open-domain QA benchmarks
demonstrate that A^2Search achieves new state-of-the-art performance. With
only a single rollout, A^2Search-7B yields an average AnsF1@1
score of 48.4% across four multi-hop benchmarks, outperforming all strong
baselines, including the substantially larger ReSearch-32B (46.2%).
Extensive analyses further show that A^2Search resolves ambiguity and
generalizes across benchmarks, highlighting that embracing ambiguity is
essential for building more reliable QA systems. Our code, data, and model
weights can be found at https://github.com/zfj1998/A2Search
A$^2$Search is an annotation-free framework that handles ambiguity in open-domain QA by detecting ambiguous questions, gathering alternative answers, and optimizing with RL, achieving state-of-the-art performance across benchmarks.
Large Language Models Reinforcement Learning open-domain question answering ambiguity standard QA benchmarks trajectory sampling evidence verification AnsF1 reward multi-hop benchmarks ReSearch-32B