GenEx: Generating an Explorable World
Understanding, navigating, and exploring the 3D physical real world has long
been a central challenge in the development of artificial intelligence. In this
work, we take a step toward this goal by introducing GenEx, a system capable of
planning complex embodied world exploration, guided by its generative
imagination that forms priors (expectations) about the surrounding
environments. GenEx generates an entire 3D-consistent imaginative environment
from as little as a single RGB image, bringing it to life through panoramic
video streams. Leveraging scalable 3D world data curated from Unreal Engine,
our generative model is rounded in the physical world. It captures a continuous
360-degree environment with little effort, offering a boundless landscape for
AI agents to explore and interact with. GenEx achieves high-quality world
generation, robust loop consistency over long trajectories, and demonstrates
strong 3D capabilities such as consistency and active 3D mapping. Powered by
generative imagination of the world, GPT-assisted agents are equipped to
perform complex embodied tasks, including both goal-agnostic exploration and
goal-driven navigation. These agents utilize predictive expectation regarding
unseen parts of the physical world to refine their beliefs, simulate different
outcomes based on potential decisions, and make more informed choices. In
summary, we demonstrate that GenEx provides a transformative platform for
advancing embodied AI in imaginative spaces and brings potential for extending
these capabilities to real-world exploration.
675bcfacd0c3aeab39880d36
Taiming Lu
claimed_verified
2024-12-13T08:56:00.579Z
656a9b9f9496f21be8271f1b
https://cdn-avatars.huggingface.co/v1/production/uploads/656a9b9f9496f21be8271f1b/e7p5kx2YNs_A1AJYSOlio.png
TaiMing
user
TaiMingLu
675bcfacd0c3aeab39880d37
Tianmin Shu
admin_assigned
2024-12-16T09:55:49.658Z
618af79af281834c95311402
/avatars/58ba086eb2309657d823b5768616eafe.svg
Tianmin Shu
user
tshu
675bcfacd0c3aeab39880d38
Junfei Xiao
claimed_verified
2024-12-16T09:42:08.190Z
64b5ba6060274cbb296d6288
/avatars/67e0343954dda6e92ed3f6e7976f9f87.svg
Junfei Xiao
user
lambertxiao
675bcfacd0c3aeab39880d39
Luoxin Ye
675bcfacd0c3aeab39880d3a
Jiahao Wang
claimed_verified
2024-12-16T09:42:10.694Z
64cbf523e3cc4a476d8291b6
/avatars/825d7665db471e46921abad3319c2846.svg
Jiahao Wang
user
jiahaoplus
675bcfacd0c3aeab39880d3b
Cheng Peng
675bcfacd0c3aeab39880d3c
Chen Wei
675bcfacd0c3aeab39880d3d
Daniel Khashabi
admin_assigned
2024-12-16T09:55:06.806Z
5f6540c65e78cc6b0ed3199d
/avatars/0280d4df417855965a0964d22766c012.svg
Daniel Khashabi
user
danyaljj
675bcfacd0c3aeab39880d3e
Rama Chellappa
675bcfacd0c3aeab39880d3f
Alan Yuille
675bcfacd0c3aeab39880d40
Jieneng Chen
admin_assigned
2024-12-16T09:54:46.355Z
660c9ac4b202fcf3892f62fa
/avatars/7314fd5f3f642096d0e37d3194f1aa7e.svg
Jieneng Chen
user
jienengchen
GenEx: Generating an Explorable World
Understanding, navigating, and exploring the 3D physical real world has long
been a central challenge in the development of artificial intelligence. In this
work, we take a step toward this goal by introducing GenEx, a system capable of
planning complex embodied world exploration, guided by its generative
imagination that forms priors (expectations) about the surrounding
environments. GenEx generates an entire 3D-consistent imaginative environment
from as little as a single RGB image, bringing it to life through panoramic
video streams. Leveraging scalable 3D world data curated from Unreal Engine,
our generative model is rounded in the physical world. It captures a continuous
360-degree environment with little effort, offering a boundless landscape for
AI agents to explore and interact with. GenEx achieves high-quality world
generation, robust loop consistency over long trajectories, and demonstrates
strong 3D capabilities such as consistency and active 3D mapping. Powered by
generative imagination of the world, GPT-assisted agents are equipped to
perform complex embodied tasks, including both goal-agnostic exploration and
goal-driven navigation. These agents utilize predictive expectation regarding
unseen parts of the physical world to refine their beliefs, simulate different
outcomes based on potential decisions, and make more informed choices. In
summary, we demonstrate that GenEx provides a transformative platform for
advancing embodied AI in imaginative spaces and brings potential for extending
these capabilities to real-world exploration.
GenEx generates 3D environments from a single image, enabling AI agents to explore and interact with a consistent, expansive space through guided generative imagination.
Generative imagination panoramic video streams Unreal Engine world generation loop consistency 3D mapping goal-agnostic exploration goal-driven navigation predictive expectation informed choices