Evolving Deeper LLM Thinking
We explore an evolutionary search strategy for scaling inference time compute
in Large Language Models. The proposed approach, Mind Evolution, uses a
language model to generate, recombine and refine candidate responses. The
proposed approach avoids the need to formalize the underlying inference problem
whenever a solution evaluator is available. Controlling for inference cost, we
find that Mind Evolution significantly outperforms other inference strategies
such as Best-of-N and Sequential Revision in natural language planning tasks.
In the TravelPlanner and Natural Plan benchmarks, Mind Evolution solves more
than 98% of the problem instances using Gemini 1.5 Pro without the use of a
formal solver.
678dc332281a0e32feb5fbfe
Kuang-Huei Lee
admin_assigned
2025-01-20T13:16:59.574Z
669fbf44200c01b737751dc5
/avatars/2023c01b2a8cc1625cafcd0b625871dc.svg
Kuang-Huei Lee
user
khlee112
678dc332281a0e32feb5fbff
Ian Fischer
admin_assigned
2025-01-20T13:18:01.647Z
64ef4f9866f36326b3ec5b8c
/avatars/9eea179e0797f952664f33e1aef21e88.svg
Ian Fischer
user
Ianfischer
678dc332281a0e32feb5fc00
Yueh-Hua Wu
678dc332281a0e32feb5fc01
Dave Marwood
678dc332281a0e32feb5fc02
Shumeet Baluja
678dc332281a0e32feb5fc03
Dale Schuurmans
678dc332281a0e32feb5fc04
Xinyun Chen
admin_assigned
2025-01-20T13:18:37.820Z
64d0268001931c60161e026a
/avatars/0ea48c47b0270b449fd6b97b495e64a6.svg
Xinyun Chen
user
xinyunchen
Evolving Deeper LLM Thinking
We explore an evolutionary search strategy for scaling inference time compute
in Large Language Models. The proposed approach, Mind Evolution, uses a
language model to generate, recombine and refine candidate responses. The
proposed approach avoids the need to formalize the underlying inference problem
whenever a solution evaluator is available. Controlling for inference cost, we
find that Mind Evolution significantly outperforms other inference strategies
such as Best-of-N and Sequential Revision in natural language planning tasks.
In the TravelPlanner and Natural Plan benchmarks, Mind Evolution solves more
than 98% of the problem instances using Gemini 1.5 Pro without the use of a
formal solver.
Mind Evolution, an evolutionary search strategy using a language model, outperforms other inference methods in natural language planning tasks by generating, recombining, and refining candidate responses.
evolutionary search strategy Large Language Models Mind Evolution language model solution evaluator Best-of-N Sequential Revision natural language planning tasks TravelPlanner Natural Plan benchmarks Gemini 1.5 Pro