Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Proving
Reasoning remains a challenging task for large language models (LLMs),
especially within the logically constrained environment of automated theorem
proving (ATP), due to sparse rewards and the vast scale of proofs. These
challenges are amplified in benchmarks like PutnamBench, which contains
university-level problems requiring complex, multi-step reasoning. To address
this, we introduce self-generated goal-conditioned MDPs (sG-MDPs), a new
framework in which agents generate and pursue their subgoals based on the
evolving proof state. Given this more structured generation of goals, the
resulting problem becomes more amenable to search. We then apply Monte Carlo
Tree Search (MCTS)-like algorithms to solve the sG-MDP, instantiating our
approach in Bourbaki (7B), a modular system that can ensemble multiple 7B LLMs
for subgoal generation and tactic synthesis. On PutnamBench, Bourbaki (7B)
solves 26 problems, achieving new state-of-the-art results with models at this
scale.
68673e349db35afc9c304d02
Matthieu Zimmer
68673e349db35afc9c304d03
Xiaotong Ji
68673e349db35afc9c304d04
Rasul Tutunov
68673e349db35afc9c304d05
Anthony Bordg
68673e349db35afc9c304d06
Jun Wang
68673e349db35afc9c304d07
Haitham Bou Ammar
Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Proving
Reasoning remains a challenging task for large language models (LLMs),
especially within the logically constrained environment of automated theorem
proving (ATP), due to sparse rewards and the vast scale of proofs. These
challenges are amplified in benchmarks like PutnamBench, which contains
university-level problems requiring complex, multi-step reasoning. To address
this, we introduce self-generated goal-conditioned MDPs (sG-MDPs), a new
framework in which agents generate and pursue their subgoals based on the
evolving proof state. Given this more structured generation of goals, the
resulting problem becomes more amenable to search. We then apply Monte Carlo
Tree Search (MCTS)-like algorithms to solve the sG-MDP, instantiating our
approach in Bourbaki (7B), a modular system that can ensemble multiple 7B LLMs
for subgoal generation and tactic synthesis. On PutnamBench, Bourbaki (7B)
solves 26 problems, achieving new state-of-the-art results with models at this
scale.
A new framework using self-generated goal-conditioned MDPs and Monte Carlo Tree Search improves automated theorem proving by generating and pursuing subgoals, achieving state-of-the-art results on PutnamBench.
self-generated goal-conditioned MDPs sG-MDPs Monte Carlo Tree Search MCTS automated theorem proving ATP Bourbaki subgoal generation tactic synthesis PutnamBench