OmniNWM: Omniscient Driving Navigation World Models
Autonomous driving world models are expected to work effectively across three
core dimensions: state, action, and reward. Existing models, however, are
typically restricted to limited state modalities, short video sequences,
imprecise action control, and a lack of reward awareness. In this paper, we
introduce OmniNWM, an omniscient panoramic navigation world model that
addresses all three dimensions within a unified framework. For state, OmniNWM
jointly generates panoramic videos of RGB, semantics, metric depth, and 3D
occupancy. A flexible forcing strategy enables high-quality long-horizon
auto-regressive generation. For action, we introduce a normalized panoramic
Plucker ray-map representation that encodes input trajectories into pixel-level
signals, enabling highly precise and generalizable control over panoramic video
generation. Regarding reward, we move beyond learning reward functions with
external image-based models: instead, we leverage the generated 3D occupancy to
directly define rule-based dense rewards for driving compliance and safety.
Extensive experiments demonstrate that OmniNWM achieves state-of-the-art
performance in video generation, control accuracy, and long-horizon stability,
while providing a reliable closed-loop evaluation framework through
occupancy-grounded rewards. Project page is available at
https://github.com/Arlo0o/OmniNWM.
68f98bbdb9b2e4ae04673789
Bohan Li
68f98bbdb9b2e4ae0467378a
Zhuang Ma
claimed_verified
2025-10-23T14:50:26.168Z
664b4ae793aed26946d7477c
/avatars/2087c10526a80f238836468b0d21f7cb.svg
Zhuang Ma
user
Genshushu
68f98bbdb9b2e4ae0467378b
Dalong Du
68f98bbdb9b2e4ae0467378c
Baorui Peng
68f98bbdb9b2e4ae0467378d
Zhujin Liang
68f98bbdb9b2e4ae0467378e
Zhenqiang Liu
68f98bbdb9b2e4ae0467378f
Chao Ma
68f98bbdb9b2e4ae04673790
Yueming Jin
68f98bbdb9b2e4ae04673791
Hao Zhao
68f98bbdb9b2e4ae04673792
Wenjun Zeng
68f98bbdb9b2e4ae04673793
Xin Jin
OmniNWM: Omniscient Driving Navigation World Models
Autonomous driving world models are expected to work effectively across three
core dimensions: state, action, and reward. Existing models, however, are
typically restricted to limited state modalities, short video sequences,
imprecise action control, and a lack of reward awareness. In this paper, we
introduce OmniNWM, an omniscient panoramic navigation world model that
addresses all three dimensions within a unified framework. For state, OmniNWM
jointly generates panoramic videos of RGB, semantics, metric depth, and 3D
occupancy. A flexible forcing strategy enables high-quality long-horizon
auto-regressive generation. For action, we introduce a normalized panoramic
Plucker ray-map representation that encodes input trajectories into pixel-level
signals, enabling highly precise and generalizable control over panoramic video
generation. Regarding reward, we move beyond learning reward functions with
external image-based models: instead, we leverage the generated 3D occupancy to
directly define rule-based dense rewards for driving compliance and safety.
Extensive experiments demonstrate that OmniNWM achieves state-of-the-art
performance in video generation, control accuracy, and long-horizon stability,
while providing a reliable closed-loop evaluation framework through
occupancy-grounded rewards. Project page is available at
https://github.com/Arlo0o/OmniNWM.
OmniNWM is a unified world model for autonomous driving that generates panoramic videos, encodes actions using Plucker ray-maps, and defines dense rewards based on 3D occupancy, achieving top performance in video generation, control, and stability.
panoramic navigation world model OmniNWM panoramic videos RGB semantics metric depth 3D occupancy auto-regressive generation normalized panoramic Plucker ray-map rule-based dense rewards driving compliance safety occupancy-grounded rewards