Optimizing Decomposition for Optimal Claim Verification
Current research on the Decompose-Then-Verify paradigm for
evaluating the factuality of long-form text typically treats decomposition and
verification in isolation, overlooking their interactions and potential
misalignment. We find that existing decomposition policies, typically
hand-crafted demonstrations, do not align well with downstream verifiers in
terms of atomicity -- a novel metric quantifying information density -- leading
to suboptimal verification results. We formulate finding the optimal
decomposition policy for optimal verification as a bilevel optimization
problem. To approximate a solution for this strongly NP-hard problem, we
propose dynamic decomposition, a reinforcement learning framework that
leverages verifier feedback to learn a policy for dynamically decomposing
claims to verifier-preferred atomicity. Experimental results show that dynamic
decomposition outperforms existing decomposition policies, improving
verification confidence by 0.07 and accuracy by 0.12 (on a 0-1 scale) on
average across varying verifiers, datasets, and atomcities of input claims.
67dc1db899d011efd7da645e
Yining Lu
claimed_verified
2025-03-20T19:17:03.538Z
642f742270daaa6e7209a2c8
/avatars/746fb840c220329f69a17905ea519322.svg
Yining Lu
user
ylu610
67dc1db899d011efd7da645f
Noah Ziems
67dc1db899d011efd7da6460
Hy Dang
67dc1db899d011efd7da6461
Meng Jiang
Optimizing Decomposition for Optimal Claim Verification
Current research on the Decompose-Then-Verify paradigm for
evaluating the factuality of long-form text typically treats decomposition and
verification in isolation, overlooking their interactions and potential
misalignment. We find that existing decomposition policies, typically
hand-crafted demonstrations, do not align well with downstream verifiers in
terms of atomicity -- a novel metric quantifying information density -- leading
to suboptimal verification results. We formulate finding the optimal
decomposition policy for optimal verification as a bilevel optimization
problem. To approximate a solution for this strongly NP-hard problem, we
propose dynamic decomposition, a reinforcement learning framework that
leverages verifier feedback to learn a policy for dynamically decomposing
claims to verifier-preferred atomicity. Experimental results show that dynamic
decomposition outperforms existing decomposition policies, improving
verification confidence by 0.07 and accuracy by 0.12 (on a 0-1 scale) on
average across varying verifiers, datasets, and atomcities of input claims.
Dynamic decomposition, a reinforcement learning approach, optimizes claim decomposition for better verification confidence and accuracy by adjusting to verifier preferences.
decomposition verification atomicity bilevel optimization reinforcement learning dynamic decomposition