PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning
  with Puzzles
This work investigates the reasoning and planning capabilities of foundation
models and their scalability in complex, dynamic environments. We introduce
PuzzlePlex, a benchmark designed to assess these capabilities through a diverse
set of puzzles. PuzzlePlex consists of 15 types of puzzles, including
deterministic and stochastic games of varying difficulty, as well as
single-player and two-player scenarios. The PuzzlePlex framework provides a
comprehensive environment for each game, and supports extensibility to generate
more challenging instances as foundation models evolve. Additionally, we
implement customized game-playing strategies for comparison. Building on this
benchmark, we develop fine-grained metrics to measure performance and conduct
an in-depth analysis of frontier foundation models across two settings:
instruction-based and code-based. Furthermore, we systematically investigate
their scaling limits. Our findings show that reasoning models outperform others
in instruction-based settings, while code-based execution presents greater
challenges but offers a scalable and efficient alternative. PuzzlePlex enables
targeted evaluation and guides future improvements in reasoning, planning, and
generalization for foundation models.
68e72c567ae125f9582e6a03
Yitao Long
claimed_verified
2025-10-09T06:49:58.156Z
641669e006af02514d0c2e9b
/avatars/4c9a7f407e7c648c60b5fc41976d80c3.svg
Yitao Lo.ong
user
Dragongon
68e72c567ae125f9582e6a04
Yuru Jiang
68e72c567ae125f9582e6a05
Hongjun Liu
claimed_verified
2025-10-10T09:32:04.600Z
646474126d621be0dd08d6d4
/avatars/5041573e603225e9de3b689efd1dd499.svg
Liu Hongjun
user
Jan150000
68e72c567ae125f9582e6a06
Yilun Zhao
68e72c567ae125f9582e6a07
Jingchen Sun
68e72c567ae125f9582e6a08
Yiqiu Shen
68e72c567ae125f9582e6a09
Chen Zhao
68e72c567ae125f9582e6a0a
Arman Cohan
68e72c567ae125f9582e6a0b
Dennis Shasha
PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning
  with Puzzles
This work investigates the reasoning and planning capabilities of foundation
models and their scalability in complex, dynamic environments. We introduce
PuzzlePlex, a benchmark designed to assess these capabilities through a diverse
set of puzzles. PuzzlePlex consists of 15 types of puzzles, including
deterministic and stochastic games of varying difficulty, as well as
single-player and two-player scenarios. The PuzzlePlex framework provides a
comprehensive environment for each game, and supports extensibility to generate
more challenging instances as foundation models evolve. Additionally, we
implement customized game-playing strategies for comparison. Building on this
benchmark, we develop fine-grained metrics to measure performance and conduct
an in-depth analysis of frontier foundation models across two settings:
instruction-based and code-based. Furthermore, we systematically investigate
their scaling limits. Our findings show that reasoning models outperform others
in instruction-based settings, while code-based execution presents greater
challenges but offers a scalable and efficient alternative. PuzzlePlex enables
targeted evaluation and guides future improvements in reasoning, planning, and
generalization for foundation models.
PuzzlePlex benchmark assesses reasoning and planning capabilities of foundation models through diverse puzzles, providing metrics and insights into their performance and scalability.
PuzzlePlex foundation models reasoning capabilities planning capabilities scalability deterministic games stochastic games single-player scenarios two-player scenarios game-playing strategies fine-grained metrics instruction-based settings code-based execution