One Life to Learn: Inferring Symbolic World Models for Stochastic
  Environments from Unguided Exploration
Symbolic world modeling requires inferring and representing an environment's
transitional dynamics as an executable program. Prior work has focused on
largely deterministic environments with abundant interaction data, simple
mechanics, and human guidance. We address a more realistic and challenging
setting, learning in a complex, stochastic environment where the agent has only
"one life" to explore a hostile environment without human guidance. We
introduce OneLife, a framework that models world dynamics through
conditionally-activated programmatic laws within a probabilistic programming
framework. Each law operates through a precondition-effect structure,
activating in relevant world states. This creates a dynamic computation graph
that routes inference and optimization only through relevant laws, avoiding
scaling challenges when all laws contribute to predictions about a complex,
hierarchical state, and enabling the learning of stochastic dynamics even with
sparse rule activation. To evaluate our approach under these demanding
constraints, we introduce a new evaluation protocol that measures (a) state
ranking, the ability to distinguish plausible future states from implausible
ones, and (b) state fidelity, the ability to generate future states that
closely resemble reality. We develop and evaluate our framework on Crafter-OO,
our reimplementation of the Crafter environment that exposes a structured,
object-oriented symbolic state and a pure transition function that operates on
that state alone. OneLife can successfully learn key environment dynamics from
minimal, unguided interaction, outperforming a strong baseline on 16 out of 23
scenarios tested. We also test OneLife's planning ability, with simulated
rollouts successfully identifying superior strategies. Our work establishes a
foundation for autonomously constructing programmatic world models of unknown,
complex environments.
68ef4328486b78128f0e35a6
Zaid Khan
68ef4328486b78128f0e35a7
Archiki Prasad
68ef4328486b78128f0e35a8
Elias Stengel-Eskin
68ef4328486b78128f0e35a9
Jaemin Cho
68ef4328486b78128f0e35aa
Mohit Bansal
One Life to Learn: Inferring Symbolic World Models for Stochastic
  Environments from Unguided Exploration
Symbolic world modeling requires inferring and representing an environment's
transitional dynamics as an executable program. Prior work has focused on
largely deterministic environments with abundant interaction data, simple
mechanics, and human guidance. We address a more realistic and challenging
setting, learning in a complex, stochastic environment where the agent has only
"one life" to explore a hostile environment without human guidance. We
introduce OneLife, a framework that models world dynamics through
conditionally-activated programmatic laws within a probabilistic programming
framework. Each law operates through a precondition-effect structure,
activating in relevant world states. This creates a dynamic computation graph
that routes inference and optimization only through relevant laws, avoiding
scaling challenges when all laws contribute to predictions about a complex,
hierarchical state, and enabling the learning of stochastic dynamics even with
sparse rule activation. To evaluate our approach under these demanding
constraints, we introduce a new evaluation protocol that measures (a) state
ranking, the ability to distinguish plausible future states from implausible
ones, and (b) state fidelity, the ability to generate future states that
closely resemble reality. We develop and evaluate our framework on Crafter-OO,
our reimplementation of the Crafter environment that exposes a structured,
object-oriented symbolic state and a pure transition function that operates on
that state alone. OneLife can successfully learn key environment dynamics from
minimal, unguided interaction, outperforming a strong baseline on 16 out of 23
scenarios tested. We also test OneLife's planning ability, with simulated
rollouts successfully identifying superior strategies. Our work establishes a
foundation for autonomously constructing programmatic world models of unknown,
complex environments.
OneLife framework models complex, stochastic environments using conditionally-activated programmatic laws within a probabilistic programming framework, enabling learning from minimal, unguided interaction and outperforming baselines in state ranking and fidelity.
conditionally-activated programmatic laws probabilistic programming framework precondition-effect structure dynamic computation graph state ranking state fidelity Crafter-OO object-oriented symbolic state pure transition function simulated rollouts