FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial
  Long-Form Question Answering
Large Language Models (LLMs) frequently hallucinate to long-form questions,
producing plausible yet factually incorrect answers. A common mitigation
strategy is to provide attribution to LLM outputs. However, existing benchmarks
primarily focus on simple attribution that retrieves supporting textual
evidence as references. We argue that in real-world scenarios such as financial
applications, attribution goes beyond reference retrieval. We introduce
FinLFQA, a benchmark designed to evaluate the ability of LLMs to generate
long-form answers to complex financial questions with reliable and nuanced
attributions. FinLFQA evaluates three critical aspects of attribution through
human annotations: (1) supporting evidence extracted from financial reports,
(2) intermediate numerical reasoning steps, and (3) domain-specific financial
knowledge that informs the reasoning process. We further provide an automatic
evaluation framework covering both answer quality and attribution quality.
Through extensive experiments on eight LLMs across multiple
attribution-generation paradigms, we find that fine-grained metrics are
important to distinguish model capabilities, that end-to-end generation
achieves comparable performance to post-hoc approaches, and that iterative
refinement only helps when guided by external feedback.
68e72d597ae125f9582e6a0e
Yitao Long
claimed_verified
2025-10-09T06:49:55.141Z
641669e006af02514d0c2e9b
/avatars/4c9a7f407e7c648c60b5fc41976d80c3.svg
Yitao Lo.ong
user
Dragongon
68e72d597ae125f9582e6a0f
Tiansheng Hu
claimed_verified
2025-10-11T13:48:58.210Z
67492b9e347c3876f22b3684
/avatars/0d80d23f7b10ce8bac689f6e8317a014.svg
Tiansheng Hu
user
HughieHu
68e72d597ae125f9582e6a10
Yilun Zhao
68e72d597ae125f9582e6a11
Arman Cohan
68e72d597ae125f9582e6a12
Chen Zhao
FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial
  Long-Form Question Answering
Large Language Models (LLMs) frequently hallucinate to long-form questions,
producing plausible yet factually incorrect answers. A common mitigation
strategy is to provide attribution to LLM outputs. However, existing benchmarks
primarily focus on simple attribution that retrieves supporting textual
evidence as references. We argue that in real-world scenarios such as financial
applications, attribution goes beyond reference retrieval. We introduce
FinLFQA, a benchmark designed to evaluate the ability of LLMs to generate
long-form answers to complex financial questions with reliable and nuanced
attributions. FinLFQA evaluates three critical aspects of attribution through
human annotations: (1) supporting evidence extracted from financial reports,
(2) intermediate numerical reasoning steps, and (3) domain-specific financial
knowledge that informs the reasoning process. We further provide an automatic
evaluation framework covering both answer quality and attribution quality.
Through extensive experiments on eight LLMs across multiple
attribution-generation paradigms, we find that fine-grained metrics are
important to distinguish model capabilities, that end-to-end generation
achieves comparable performance to post-hoc approaches, and that iterative
refinement only helps when guided by external feedback.
FinLFQA evaluates LLMs' ability to provide reliable and nuanced attributions in long-form financial question answering through human and automatic assessments.
Large Language Models LLMs hallucination attribution FinLFQA supporting evidence intermediate numerical reasoning domain-specific financial knowledge end-to-end generation post-hoc approaches iterative refinement