Self-Steering Language Models
While test-time reasoning enables language models to tackle complex tasks,
searching or planning in natural language can be slow, costly, and error-prone.
But even when LMs struggle to emulate the precise reasoning steps needed to
solve a problem, they often excel at describing its abstract structure--both
how to verify solutions and how to search for them. This paper introduces
DisCIPL, a method for "self-steering" LMs where a Planner model generates a
task-specific inference program that is executed by a population of Follower
models. Our approach equips LMs with the ability to write recursive search
procedures that guide LM inference, enabling new forms of verifiable and
efficient reasoning. When instantiated with a small Follower (e.g.,
Llama-3.2-1B), DisCIPL matches (and sometimes outperforms) much larger models,
including GPT-4o and o1, on challenging constrained generation tasks. In
decoupling planning from execution, our work opens up a design space of
highly-parallelized Monte Carlo inference strategies that outperform standard
best-of-N sampling, require no finetuning, and can be implemented automatically
by existing LMs.
67f7823da630bcdabbd8b3eb
Gabriel Grand
67f7823da630bcdabbd8b3ec
Joshua B. Tenenbaum
67f7823da630bcdabbd8b3ed
Vikash K. Mansinghka
67f7823da630bcdabbd8b3ee
Alexander K. Lew
extracted_pending
2025-04-10T08:33:02.433Z
673cc08370644bb836283fec
/avatars/b8c7f1a10ddf76dcd06398c59f553b61.svg
Alexander Lew
user
alexanderlew
67f7823da630bcdabbd8b3ef
Jacob Andreas
Self-Steering Language Models
While test-time reasoning enables language models to tackle complex tasks,
searching or planning in natural language can be slow, costly, and error-prone.
But even when LMs struggle to emulate the precise reasoning steps needed to
solve a problem, they often excel at describing its abstract structure--both
how to verify solutions and how to search for them. This paper introduces
DisCIPL, a method for "self-steering" LMs where a Planner model generates a
task-specific inference program that is executed by a population of Follower
models. Our approach equips LMs with the ability to write recursive search
procedures that guide LM inference, enabling new forms of verifiable and
efficient reasoning. When instantiated with a small Follower (e.g.,
Llama-3.2-1B), DisCIPL matches (and sometimes outperforms) much larger models,
including GPT-4o and o1, on challenging constrained generation tasks. In
decoupling planning from execution, our work opens up a design space of
highly-parallelized Monte Carlo inference strategies that outperform standard
best-of-N sampling, require no finetuning, and can be implemented automatically
by existing LMs.
DisCIPL, a method combining a Planner and Follower model, enables efficient and verifiable reasoning in language models by generating task-specific inference programs.
test-time reasoning language models Planner model Follower models recursive search procedures verifiable reasoning constrained generation tasks Monte Carlo inference strategies best-of-N sampling existing LMs