Evolving Deeper LLM Thinking
We explore an evolutionary search strategy for scaling inference time compute
in Large Language Models. The proposed approach, Mind Evolution, uses a
language model to generate, recombine and refine candidate responses. The
proposed approach avoids the need to formalize the underlying inference problem
whenever a solution evaluator is available. Controlling for inference cost, we
find that Mind Evolution significantly outperforms other inference strategies
such as Best-of-N and Sequential Revision in natural language planning tasks.
In the TravelPlanner and Natural Plan benchmarks, Mind Evolution solves more
than 98% of the problem instances using Gemini 1.5 Pro without the use of a
formal solver.
678dc332281a0e32feb5fbfe
Kuang-Huei Lee
admin_assigned
2025-01-20T13:16:59.574Z
669fbf44200c01b737751dc5
/avatars/2023c01b2a8cc1625cafcd0b625871dc.svg
Kuang-Huei Lee
user
khlee112
678dc332281a0e32feb5fbff
Ian Fischer
admin_assigned
2025-01-20T13:18:01.647Z
64ef4f9866f36326b3ec5b8c
/avatars/9eea179e0797f952664f33e1aef21e88.svg
Ian Fischer
user
Ianfischer
678dc332281a0e32feb5fc00
Yueh-Hua Wu
678dc332281a0e32feb5fc01
Dave Marwood
678dc332281a0e32feb5fc02
Shumeet Baluja
678dc332281a0e32feb5fc03
Dale Schuurmans
678dc332281a0e32feb5fc04
Xinyun Chen
admin_assigned
2025-01-20T13:18:37.820Z
64d0268001931c60161e026a
/avatars/0ea48c47b0270b449fd6b97b495e64a6.svg
Xinyun Chen
user
xinyunchen
Evolving Deeper LLM Thinking
We explore an evolutionary search strategy for scaling inference time compute
in Large Language Models. The proposed approach, Mind Evolution, uses a
language model to generate, recombine and refine candidate responses. The
proposed approach avoids the need to formalize the underlying inference problem
whenever a solution evaluator is available. Controlling for inference cost, we
find that Mind Evolution significantly outperforms other inference strategies
such as Best-of-N and Sequential Revision in natural language planning tasks.
In the TravelPlanner and Natural Plan benchmarks, Mind Evolution solves more
than 98% of the problem instances using Gemini 1.5 Pro without the use of a
formal solver.
evolutionary search strategy inference time compute Large Language Models Mind Evolution candidate responses solution evaluator natural language planning tasks TravelPlanner Natural Plan benchmarks Gemini 1.5 Pro formal solver