ARR: Question Answering with Large Language Models via Analyzing,
  Retrieving, and Reasoning
Large language models (LLMs) achieve remarkable performance on challenging
benchmarks that are often structured as multiple-choice question-answering (QA)
tasks. Zero-shot Chain-of-Thought (CoT) prompting enhances reasoning in LLMs
but provides only vague and generic guidance ("think step by step"). This paper
introduces ARR, an intuitive and effective zero-shot prompting method that
explicitly incorporates three key steps in QA solving: analyzing the intent of
the question, retrieving relevant information, and reasoning step by step.
Comprehensive experiments across diverse and challenging QA tasks demonstrate
that ARR consistently improves the Baseline (without ARR prompting) and
outperforms CoT. Ablation and case studies further validate the positive
contributions of each component: analyzing, retrieving, and reasoning. Notably,
intent analysis plays a vital role in ARR. Additionally, extensive evaluations
across various model sizes, LLM series, and generation settings solidify the
effectiveness, robustness, and generalizability of ARR.
67a9b911b1f5eece682d7961
Yuwei Yin
claimed_verified
2025-02-10T09:49:32.672Z
64510a21f800611f94f0d9f8
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/lOeHK9Bvt3IXcB7Urx6jZ.jpeg
Yuwei Yin
user
yuweiyin
67a9b911b1f5eece682d7962
Giuseppe Carenini
ARR: Question Answering with Large Language Models via Analyzing,
  Retrieving, and Reasoning
Large language models (LLMs) achieve remarkable performance on challenging
benchmarks that are often structured as multiple-choice question-answering (QA)
tasks. Zero-shot Chain-of-Thought (CoT) prompting enhances reasoning in LLMs
but provides only vague and generic guidance ("think step by step"). This paper
introduces ARR, an intuitive and effective zero-shot prompting method that
explicitly incorporates three key steps in QA solving: analyzing the intent of
the question, retrieving relevant information, and reasoning step by step.
Comprehensive experiments across diverse and challenging QA tasks demonstrate
that ARR consistently improves the Baseline (without ARR prompting) and
outperforms CoT. Ablation and case studies further validate the positive
contributions of each component: analyzing, retrieving, and reasoning. Notably,
intent analysis plays a vital role in ARR. Additionally, extensive evaluations
across various model sizes, LLM series, and generation settings solidify the
effectiveness, robustness, and generalizability of ARR.
ARR, a zero-shot prompting method for question-answering tasks, incorporates intent analysis, retrieval, and reasoning to enhance performance beyond zero-shot Chain-of-Thought prompting.
Large language models zero-shot Chain-of-Thought zero-shot prompting intent analysis retrieval reasoning question-answering comprehensive experiments ablation studies case studies model sizes LLM series generation settings