RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
The reasoning steps generated by LLMs might be incomplete, as they mimic
logical leaps common in everyday communication found in their pre-training
data: underlying rationales are frequently left implicit (unstated). To address
this challenge, we introduce RATIONALYST, a model for process-supervision of
reasoning based on pre-training on a vast collection of rationale annotations
extracted from unlabeled data. We extract 79k rationales from web-scale
unlabelled dataset (the Pile) and a combination of reasoning datasets with
minimal human intervention. This web-scale pre-training for reasoning allows
RATIONALYST to consistently generalize across diverse reasoning tasks,
including mathematical, commonsense, scientific, and logical reasoning.
Fine-tuned from LLaMa-3-8B, RATIONALYST improves the accuracy of reasoning by
an average of 3.9% on 7 representative reasoning benchmarks. It also
demonstrates superior performance compared to significantly larger verifiers
like GPT-4 and similarly sized models fine-tuned on matching training sets.
66fe079fe9e5b5c3e49e309e
Dongwei Jiang
admin_assigned
2024-10-03T09:15:24.379Z
63eba2774dcaaf087638e3d6
https://cdn-avatars.huggingface.co/v1/production/uploads/1676386919413-noauth.jpeg
Jiang
user
Dongwei
66fe079fe9e5b5c3e49e309f
Guoxuan Wang
66fe079fe9e5b5c3e49e30a0
Yining Lu
admin_assigned
2024-10-03T09:22:30.977Z
642f742270daaa6e7209a2c8
/avatars/746fb840c220329f69a17905ea519322.svg
Yining Lu
user
ylu610
66fe079fe9e5b5c3e49e30a1
Andrew Wang
66fe079fe9e5b5c3e49e30a2
Jingyu Zhang
claimed_verified
2024-10-04T07:32:14.554Z
62fb40b59af1d16bc0ac60f4
/avatars/03ff66a419db8f2bc8e89a3b47aaaeac.svg
Jack Zhang
user
jackzhang
66fe079fe9e5b5c3e49e30a3
Chuyu Liu
admin_assigned
2024-10-03T09:20:21.699Z
66240a576046dd4b8586a604
/avatars/6b4c5fbdaafd6f42d8f4581d0f48d2c0.svg
Chuyu Liu
user
ChuyuLiu
66fe079fe9e5b5c3e49e30a4
Benjamin Van Durme
66fe079fe9e5b5c3e49e30a5
Daniel Khashabi
admin_assigned
2024-10-03T09:20:05.299Z
5f6540c65e78cc6b0ed3199d
/avatars/0280d4df417855965a0964d22766c012.svg
Daniel Khashabi
user
danyaljj
RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
The reasoning steps generated by LLMs might be incomplete, as they mimic
logical leaps common in everyday communication found in their pre-training
data: underlying rationales are frequently left implicit (unstated). To address
this challenge, we introduce RATIONALYST, a model for process-supervision of
reasoning based on pre-training on a vast collection of rationale annotations
extracted from unlabeled data. We extract 79k rationales from web-scale
unlabelled dataset (the Pile) and a combination of reasoning datasets with
minimal human intervention. This web-scale pre-training for reasoning allows
RATIONALYST to consistently generalize across diverse reasoning tasks,
including mathematical, commonsense, scientific, and logical reasoning.
Fine-tuned from LLaMa-3-8B, RATIONALYST improves the accuracy of reasoning by
an average of 3.9% on 7 representative reasoning benchmarks. It also
demonstrates superior performance compared to significantly larger verifiers
like GPT-4 and similarly sized models fine-tuned on matching training sets.
RATIONALYST, a process-supervised reasoning model pre-trained on vast web-scale rationale annotations, enhances reasoning accuracy across diverse tasks compared to larger models.
RATIONALYST process-supervision rationale annotations pre-training reasoning tasks mathematical reasoning commonsense reasoning scientific reasoning logical reasoning fine-tuned LLaMa-3-8B accuracy reasoning benchmarks GPT-4