Learning to Reason for Hallucination Span Detection
Large language models (LLMs) often generate hallucinations -- unsupported
content that undermines reliability. While most prior works frame hallucination
detection as a binary task, many real-world applications require identifying
hallucinated spans, which is a multi-step decision making process. This
naturally raises the question of whether explicit reasoning can help the
complex task of detecting hallucination spans. To answer this question, we
first evaluate pretrained models with and without Chain-of-Thought (CoT)
reasoning, and show that CoT reasoning has the potential to generate at least
one correct answer when sampled multiple times. Motivated by this, we propose
RL4HS, a reinforcement learning framework that incentivizes reasoning with a
span-level reward function. RL4HS builds on Group Relative Policy Optimization
and introduces Class-Aware Policy Optimization to mitigate reward imbalance
issue. Experiments on the RAGTruth benchmark (summarization, question
answering, data-to-text) show that RL4HS surpasses pretrained reasoning models
and supervised fine-tuning, demonstrating the necessity of reinforcement
learning with span-level rewards for detecting hallucination spans.
68df5d52df49fb0df1e03d7a
Hsuan Su
claimed_verified
2025-10-03T12:28:53.649Z
608abf1272b50b02c4b02865
https://cdn-avatars.huggingface.co/v1/production/uploads/1619708309549-608abf1272b50b02c4b02865.jpeg
Hsuan Su
user
jacksukk
68df5d52df49fb0df1e03d7b
Ting-Yao Hu
68df5d52df49fb0df1e03d7c
Hema Swetha Koppula
68df5d52df49fb0df1e03d7d
Kundan Krishna
68df5d52df49fb0df1e03d7e
Hadi Pouransari
68df5d52df49fb0df1e03d7f
Cheng-Yu Hsieh
68df5d52df49fb0df1e03d80
Cem Koc
68df5d52df49fb0df1e03d81
Joseph Yitan Cheng
68df5d52df49fb0df1e03d82
Oncel Tuzel
68df5d52df49fb0df1e03d83
Raviteja Vemulapalli
Learning to Reason for Hallucination Span Detection
Large language models (LLMs) often generate hallucinations -- unsupported
content that undermines reliability. While most prior works frame hallucination
detection as a binary task, many real-world applications require identifying
hallucinated spans, which is a multi-step decision making process. This
naturally raises the question of whether explicit reasoning can help the
complex task of detecting hallucination spans. To answer this question, we
first evaluate pretrained models with and without Chain-of-Thought (CoT)
reasoning, and show that CoT reasoning has the potential to generate at least
one correct answer when sampled multiple times. Motivated by this, we propose
RL4HS, a reinforcement learning framework that incentivizes reasoning with a
span-level reward function. RL4HS builds on Group Relative Policy Optimization
and introduces Class-Aware Policy Optimization to mitigate reward imbalance
issue. Experiments on the RAGTruth benchmark (summarization, question
answering, data-to-text) show that RL4HS surpasses pretrained reasoning models
and supervised fine-tuning, demonstrating the necessity of reinforcement
learning with span-level rewards for detecting hallucination spans.
A reinforcement learning framework with span-level rewards improves hallucination span detection in large language models by incentivizing reasoning.
Chain-of-Thought reinforcement learning span-level reward function Group Relative Policy Optimization Class-Aware Policy Optimization RAGTruth benchmark summarization question answering data-to-text hallucination span detection