GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action
  Control
Recent advancements in world models have revolutionized dynamic environment
simulation, allowing systems to foresee future states and assess potential
actions. In autonomous driving, these capabilities help vehicles anticipate the
behavior of other road users, perform risk-aware planning, accelerate training
in simulation, and adapt to novel scenarios, thereby enhancing safety and
reliability. Current approaches exhibit deficiencies in maintaining robust 3D
geometric consistency or accumulating artifacts during occlusion handling, both
critical for reliable safety assessment in autonomous navigation tasks. To
address this, we introduce GeoDrive, which explicitly integrates robust 3D
geometry conditions into driving world models to enhance spatial understanding
and action controllability. Specifically, we first extract a 3D representation
from the input frame and then obtain its 2D rendering based on the
user-specified ego-car trajectory. To enable dynamic modeling, we propose a
dynamic editing module during training to enhance the renderings by editing the
positions of the vehicles. Extensive experiments demonstrate that our method
significantly outperforms existing models in both action accuracy and 3D
spatial awareness, leading to more realistic, adaptable, and reliable scene
modeling for safer autonomous driving. Additionally, our model can generalize
to novel trajectories and offers interactive scene editing capabilities, such
as object editing and object trajectory control.
68391dbb0653b6a3441a7f7e
Anthony Chen
claimed_verified
2025-05-30T06:53:38.940Z
6311d9ee04f842f79916158c
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/PvOYgDEpe0x5ExLyj1BK-.png
chen
user
antonio-c
68391dbb0653b6a3441a7f7f
Wenzhao Zheng
68391dbb0653b6a3441a7f80
Yida Wang
claimed_verified
2025-05-30T06:53:36.769Z
647068944be5cf1f33491cb9
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Aqudl2PTSKPAylBhlccWr.png
Yida Wang
user
wangyida
68391dbb0653b6a3441a7f81
Xueyang Zhang
68391dbb0653b6a3441a7f82
Kun Zhan
68391dbb0653b6a3441a7f83
Peng Jia
68391dbb0653b6a3441a7f84
Kurt Keutzer
68391dbb0653b6a3441a7f85
Shanghang Zhang
GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action
  Control
Recent advancements in world models have revolutionized dynamic environment
simulation, allowing systems to foresee future states and assess potential
actions. In autonomous driving, these capabilities help vehicles anticipate the
behavior of other road users, perform risk-aware planning, accelerate training
in simulation, and adapt to novel scenarios, thereby enhancing safety and
reliability. Current approaches exhibit deficiencies in maintaining robust 3D
geometric consistency or accumulating artifacts during occlusion handling, both
critical for reliable safety assessment in autonomous navigation tasks. To
address this, we introduce GeoDrive, which explicitly integrates robust 3D
geometry conditions into driving world models to enhance spatial understanding
and action controllability. Specifically, we first extract a 3D representation
from the input frame and then obtain its 2D rendering based on the
user-specified ego-car trajectory. To enable dynamic modeling, we propose a
dynamic editing module during training to enhance the renderings by editing the
positions of the vehicles. Extensive experiments demonstrate that our method
significantly outperforms existing models in both action accuracy and 3D
spatial awareness, leading to more realistic, adaptable, and reliable scene
modeling for safer autonomous driving. Additionally, our model can generalize
to novel trajectories and offers interactive scene editing capabilities, such
as object editing and object trajectory control.
GeoDrive integrates robust 3D geometry into driving world models to improve spatial understanding and action controllability in autonomous navigation, enhancing safety and reliability.
world models dynamic environment simulation autonomous driving 3D geometric consistency occlusion handling 3D representation 2D rendering ego-car trajectory dynamic editing module action accuracy 3D spatial awareness scene modeling object editing object trajectory control