Fin-PRM: A Domain-Specialized Process Reward Model for Financial
  Reasoning in Large Language Models
Process Reward Models (PRMs) have emerged as a promising framework for
supervising intermediate reasoning in large language models (LLMs), yet
existing PRMs are primarily trained on general or Science, Technology,
Engineering, and Mathematics (STEM) domains and fall short in domain-specific
contexts such as finance, where reasoning is more structured, symbolic, and
sensitive to factual and regulatory correctness. We introduce Fin-PRM,
a domain-specialized, trajectory-aware PRM tailored to evaluate intermediate
reasoning steps in financial tasks. Fin-PRM integrates step-level and
trajectory-level reward supervision, enabling fine-grained evaluation of
reasoning traces aligned with financial logic. We apply Fin-PRM in both offline
and online reward learning settings, supporting three key applications: (i)
selecting high-quality reasoning trajectories for distillation-based supervised
fine-tuning, (ii) providing dense process-level rewards for reinforcement
learning, and (iii) guiding reward-informed Best-of-N inference at test time.
Experimental results on financial reasoning benchmarks, including CFLUE and
FinQA, demonstrate that Fin-PRM consistently outperforms general-purpose PRMs
and strong domain baselines in trajectory selection quality. Downstream models
trained with Fin-PRM yield substantial improvements with baselines, with gains
of 12.9\% in supervised learning, 5.2\% in reinforcement learning, and 5.1\% in
test-time performance. These findings highlight the value of domain-specialized
reward modeling for aligning LLMs with expert-level financial reasoning. Our
project resources will be available at https://github.com/aliyun/qwen-dianjin.
68a7cd6f39413c456c05b0a4
Yuanchen Zhou
68a7cd6f39413c456c05b0a5
Shuo Jiang
68a7cd6f39413c456c05b0a6
Jie Zhu
claimed_verified
2025-08-22T07:21:59.230Z
642656cbad1e3b0e6e91b752
/avatars/3bf0ee15fd528e09b2b889f5cce3cbd0.svg
Jie Zhu
user
amazingj
68a7cd6f39413c456c05b0a7
Junhui Li
68a7cd6f39413c456c05b0a8
Lifan Guo
68a7cd6f39413c456c05b0a9
Feng Chen
68a7cd6f39413c456c05b0aa
Chi Zhang
Fin-PRM: A Domain-Specialized Process Reward Model for Financial
  Reasoning in Large Language Models
Process Reward Models (PRMs) have emerged as a promising framework for
supervising intermediate reasoning in large language models (LLMs), yet
existing PRMs are primarily trained on general or Science, Technology,
Engineering, and Mathematics (STEM) domains and fall short in domain-specific
contexts such as finance, where reasoning is more structured, symbolic, and
sensitive to factual and regulatory correctness. We introduce Fin-PRM,
a domain-specialized, trajectory-aware PRM tailored to evaluate intermediate
reasoning steps in financial tasks. Fin-PRM integrates step-level and
trajectory-level reward supervision, enabling fine-grained evaluation of
reasoning traces aligned with financial logic. We apply Fin-PRM in both offline
and online reward learning settings, supporting three key applications: (i)
selecting high-quality reasoning trajectories for distillation-based supervised
fine-tuning, (ii) providing dense process-level rewards for reinforcement
learning, and (iii) guiding reward-informed Best-of-N inference at test time.
Experimental results on financial reasoning benchmarks, including CFLUE and
FinQA, demonstrate that Fin-PRM consistently outperforms general-purpose PRMs
and strong domain baselines in trajectory selection quality. Downstream models
trained with Fin-PRM yield substantial improvements with baselines, with gains
of 12.9\% in supervised learning, 5.2\% in reinforcement learning, and 5.1\% in
test-time performance. These findings highlight the value of domain-specialized
reward modeling for aligning LLMs with expert-level financial reasoning. Our
project resources will be available at https://github.com/aliyun/qwen-dianjin.
Fin-PRM, a domain-specialized reward model for finance, enhances intermediate reasoning in LLMs through step-level and trajectory-level supervision, improving performance in supervised learning, reinforcement learning, and test-time inference.
Process Reward Models PRMs large language models LLMs domain-specific finance structured reasoning symbolic reasoning factual correctness regulatory correctness trajectory-aware step-level supervision trajectory-level supervision reasoning traces financial logic distillation-based supervised fine-tuning dense process-level rewards reinforcement learning reward-informed Best-of-N inference CFLUE FinQA trajectory selection quality supervised learning test-time performance