OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks
  with Reinforcement Fine-Tuning
OpenAI's recent introduction of Reinforcement Fine-Tuning (RFT) showcases the
potential of reasoning foundation model and offers a new paradigm for
fine-tuning beyond simple pattern imitation. This technical report presents
OpenRFT, our attempt to fine-tune generalist reasoning models for
domain-specific tasks under the same settings as RFT. OpenRFT addresses two key
challenges of lacking reasoning step data and the limited quantity of training
samples, by leveraging the domain-specific samples in three ways: question
augmentation, synthesizing reasoning-process data, and few-shot ICL. The
evaluation is conducted on SciKnowEval, where OpenRFT achieves notable
performance gains with only 100 domain-specific samples for each task. More
experimental results will be updated continuously in later versions. Source
codes, datasets, and models are disclosed at:
https://github.com/ADaM-BJTU/OpenRFT
676a85a619a646a97e21e8e5
Yuxiang Zhang
claimed_verified
2024-12-30T19:34:36.448Z
645b4a2978730bcc103dfe4d
/avatars/de544de899897fd0a83506ff287123bc.svg
Yuxiang Zhang
user
TokerZ
676a85a619a646a97e21e8e6
Yuqi Yang
676a85a619a646a97e21e8e7
Jiangming Shu
676a85a619a646a97e21e8e8
Yuhang Wang
claimed_verified
2025-01-14T08:30:16.733Z
6492a388bb63613026a34ccd
/avatars/83352981ef3858db3655826a869ba730.svg
Yuhang Wang
user
Rykeryh
676a85a619a646a97e21e8e9
Jinlin Xiao
676a85a619a646a97e21e8ea
Jitao Sang
OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks
  with Reinforcement Fine-Tuning
OpenAI's recent introduction of Reinforcement Fine-Tuning (RFT) showcases the
potential of reasoning foundation model and offers a new paradigm for
fine-tuning beyond simple pattern imitation. This technical report presents
OpenRFT, our attempt to fine-tune generalist reasoning models for
domain-specific tasks under the same settings as RFT. OpenRFT addresses two key
challenges of lacking reasoning step data and the limited quantity of training
samples, by leveraging the domain-specific samples in three ways: question
augmentation, synthesizing reasoning-process data, and few-shot ICL. The
evaluation is conducted on SciKnowEval, where OpenRFT achieves notable
performance gains with only 100 domain-specific samples for each task. More
experimental results will be updated continuously in later versions. Source
codes, datasets, and models are disclosed at:
https://github.com/ADaM-BJTU/OpenRFT
Reinforcement Fine-Tuning (RFT) generalist reasoning models domain-specific tasks question augmentation synthesizing reasoning-process data few-shot ICL SciKnowEval