Diffusion World Model
We introduce Diffusion World Model (DWM), a conditional diffusion model
capable of predicting multistep future states and rewards concurrently. As
opposed to traditional one-step dynamics models, DWM offers long-horizon
predictions in a single forward pass, eliminating the need for recursive
quires. We integrate DWM into model-based value estimation, where the
short-term return is simulated by future trajectories sampled from DWM. In the
context of offline reinforcement learning, DWM can be viewed as a conservative
value regularization through generative modeling. Alternatively, it can be seen
as a data source that enables offline Q-learning with synthetic data. Our
experiments on the D4RL dataset confirm the robustness of DWM to long-horizon
simulation. In terms of absolute performance, DWM significantly surpasses
one-step dynamics models with a 44% performance gain, and achieves
state-of-the-art performance.
65c3062182d1feaf1b21a643
Zihan Ding
extracted_pending
2024-02-07T04:25:07.178Z
627be1464d0858f00350ebba
/avatars/d2ecacf46f416f46a0df29e9153b3f11.svg
Zihan Ding
user
quantumiracle
65c3062182d1feaf1b21a644
Amy Zhang
65c3062182d1feaf1b21a645
Yuandong Tian
65c3062182d1feaf1b21a646
Qinqing Zheng
claimed_verified
2025-02-07T09:59:36.169Z
64d27579dafee18faf9308ac
/avatars/8914a47244017c3541d3d5ac5b2d0372.svg
Qinqing Zheng
user
goodsleep
Diffusion World Model
We introduce Diffusion World Model (DWM), a conditional diffusion model
capable of predicting multistep future states and rewards concurrently. As
opposed to traditional one-step dynamics models, DWM offers long-horizon
predictions in a single forward pass, eliminating the need for recursive
quires. We integrate DWM into model-based value estimation, where the
short-term return is simulated by future trajectories sampled from DWM. In the
context of offline reinforcement learning, DWM can be viewed as a conservative
value regularization through generative modeling. Alternatively, it can be seen
as a data source that enables offline Q-learning with synthetic data. Our
experiments on the D4RL dataset confirm the robustness of DWM to long-horizon
simulation. In terms of absolute performance, DWM significantly surpasses
one-step dynamics models with a 44% performance gain, and achieves
state-of-the-art performance.
A conditional diffusion model (DWM) is introduced for long-horizon predictions in model-based reinforcement learning, surpassing traditional one-step models and achieving state-of-the-art performance.
Diffusion World Model conditional diffusion model long-horizon predictions single forward pass model-based value estimation offline reinforcement learning generative modeling synthetic data D4RL dataset one-step dynamics models