VerIF: Verification Engineering for Reinforcement Learning in
  Instruction Following
Reinforcement learning with verifiable rewards (RLVR) has become a key
technique for enhancing large language models (LLMs), with verification
engineering playing a central role. However, best practices for RL in
instruction following remain underexplored. In this work, we explore the
verification challenge in RL for instruction following and propose VerIF, a
verification method that combines rule-based code verification with LLM-based
verification from a large reasoning model (e.g., QwQ-32B). To support this
approach, we construct a high-quality instruction-following dataset,
VerInstruct, containing approximately 22,000 instances with associated
verification signals. We apply RL training with VerIF to two models, achieving
significant improvements across several representative instruction-following
benchmarks. The trained models reach state-of-the-art performance among models
of comparable size and generalize well to unseen constraints. We further
observe that their general capabilities remain unaffected, suggesting that RL
with VerIF can be integrated into existing RL recipes to enhance overall model
performance. We have released our datasets, codes, and models to facilitate
future research at https://github.com/THU-KEG/VerIF.
684ae26adbd21a9cc27b1177
Hao Peng
claimed_verified
2025-06-13T07:40:06.456Z
625a5446f1063e7085d5178a
/avatars/5e78186f13f74b14e01583e06ff6c4dc.svg
Hao Peng
user
Wesleythu
684ae26adbd21a9cc27b1178
Yunjia Qi
684ae26adbd21a9cc27b1179
Xiaozhi Wang
684ae26adbd21a9cc27b117a
Bin Xu
684ae26adbd21a9cc27b117b
Lei Hou
684ae26adbd21a9cc27b117c
Juanzi Li
VerIF: Verification Engineering for Reinforcement Learning in
  Instruction Following
Reinforcement learning with verifiable rewards (RLVR) has become a key
technique for enhancing large language models (LLMs), with verification
engineering playing a central role. However, best practices for RL in
instruction following remain underexplored. In this work, we explore the
verification challenge in RL for instruction following and propose VerIF, a
verification method that combines rule-based code verification with LLM-based
verification from a large reasoning model (e.g., QwQ-32B). To support this
approach, we construct a high-quality instruction-following dataset,
VerInstruct, containing approximately 22,000 instances with associated
verification signals. We apply RL training with VerIF to two models, achieving
significant improvements across several representative instruction-following
benchmarks. The trained models reach state-of-the-art performance among models
of comparable size and generalize well to unseen constraints. We further
observe that their general capabilities remain unaffected, suggesting that RL
with VerIF can be integrated into existing RL recipes to enhance overall model
performance. We have released our datasets, codes, and models to facilitate
future research at https://github.com/THU-KEG/VerIF.
VerIF, a hybrid verification method combining rule-based and LLM-based approaches, enhances instruction-following RL with significant performance improvements and generalization.
reinforcement learning verifiable rewards RLVR large language models LLMs rule-based code verification QwQ-32B instruction-following VerInstruct RL training instruction-following benchmarks state-of-the-art performance existing RL recipes