Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
We present Habitat 3.0: a simulation platform for studying collaborative
human-robot tasks in home environments. Habitat 3.0 offers contributions across
three dimensions: (1) Accurate humanoid simulation: addressing challenges in
modeling complex deformable bodies and diversity in appearance and motion, all
while ensuring high simulation speed. (2) Human-in-the-loop infrastructure:
enabling real human interaction with simulated robots via mouse/keyboard or a
VR interface, facilitating evaluation of robot policies with human input. (3)
Collaborative tasks: studying two collaborative tasks, Social Navigation and
Social Rearrangement. Social Navigation investigates a robot's ability to
locate and follow humanoid avatars in unseen environments, whereas Social
Rearrangement addresses collaboration between a humanoid and robot while
rearranging a scene. These contributions allow us to study end-to-end learned
and heuristic baselines for human-robot collaboration in-depth, as well as
evaluate them with humans in the loop. Our experiments demonstrate that learned
robot policies lead to efficient task completion when collaborating with unseen
humanoid agents and human partners that might exhibit behaviors that the robot
has not seen before. Additionally, we observe emergent behaviors during
collaborative task execution, such as the robot yielding space when obstructing
a humanoid agent, thereby allowing the effective completion of the task by the
humanoid agent. Furthermore, our experiments using the human-in-the-loop tool
demonstrate that our automated evaluation with humanoids can provide an
indication of the relative ordering of different policies when evaluated with
real human collaborators. Habitat 3.0 unlocks interesting new features in
simulators for Embodied AI, and we hope it paves the way for a new frontier of
embodied human-AI interaction capabilities.
6537dcecd325b3f02fca4e4f
Xavier Puig
6537dcecd325b3f02fca4e50
Eric Undersander
6537dcecd325b3f02fca4e51
Andrew Szot
6537dcecd325b3f02fca4e52
Mikael Dallaire Cote
6537dcecd325b3f02fca4e53
Tsung-Yen Yang
6537dcecd325b3f02fca4e54
Ruslan Partsey
6537dcecd325b3f02fca4e55
Ruta Desai
6537dcecd325b3f02fca4e56
Alexander William Clegg
6537dcecd325b3f02fca4e57
Michal Hlavac
6537dcecd325b3f02fca4e58
So Yeon Min
6537dcecd325b3f02fca4e59
Vladimír Vondruš
6537dcecd325b3f02fca4e5a
Theophile Gervet
6537dcecd325b3f02fca4e5b
Vincent-Pierre Berges
6537dcecd325b3f02fca4e5c
John M. Turner
claimed_verified
2024-01-12T16:34:36.738Z
63ee68535f1300034dde7848
https://cdn-avatars.huggingface.co/v1/production/uploads/1676570521120-63ee68535f1300034dde7848.jpeg
John Turner
user
JohnTurner
6537dcecd325b3f02fca4e5d
Oleksandr Maksymets
6537dcecd325b3f02fca4e5e
Zsolt Kira
6537dcecd325b3f02fca4e5f
Mrinal Kalakrishnan
6537dcecd325b3f02fca4e60
Jitendra Malik
6537dcecd325b3f02fca4e61
Devendra Singh Chaplot
6537dcecd325b3f02fca4e62
Unnat Jain
6537dcecd325b3f02fca4e63
Dhruv Batra
6537dcecd325b3f02fca4e64
Akshara Rai
6537dcecd325b3f02fca4e65
Roozbeh Mottaghi
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
We present Habitat 3.0: a simulation platform for studying collaborative
human-robot tasks in home environments. Habitat 3.0 offers contributions across
three dimensions: (1) Accurate humanoid simulation: addressing challenges in
modeling complex deformable bodies and diversity in appearance and motion, all
while ensuring high simulation speed. (2) Human-in-the-loop infrastructure:
enabling real human interaction with simulated robots via mouse/keyboard or a
VR interface, facilitating evaluation of robot policies with human input. (3)
Collaborative tasks: studying two collaborative tasks, Social Navigation and
Social Rearrangement. Social Navigation investigates a robot's ability to
locate and follow humanoid avatars in unseen environments, whereas Social
Rearrangement addresses collaboration between a humanoid and robot while
rearranging a scene. These contributions allow us to study end-to-end learned
and heuristic baselines for human-robot collaboration in-depth, as well as
evaluate them with humans in the loop. Our experiments demonstrate that learned
robot policies lead to efficient task completion when collaborating with unseen
humanoid agents and human partners that might exhibit behaviors that the robot
has not seen before. Additionally, we observe emergent behaviors during
collaborative task execution, such as the robot yielding space when obstructing
a humanoid agent, thereby allowing the effective completion of the task by the
humanoid agent. Furthermore, our experiments using the human-in-the-loop tool
demonstrate that our automated evaluation with humanoids can provide an
indication of the relative ordering of different policies when evaluated with
real human collaborators. Habitat 3.0 unlocks interesting new features in
simulators for Embodied AI, and we hope it paves the way for a new frontier of
embodied human-AI interaction capabilities.
Habitat 3.0 provides a simulation platform for studying collaborative human-robot tasks with accurate humanoid simulation, human-in-the-loop infrastructure, and support for social navigation and rearrangement tasks.
humanoid simulation deformable bodies motion diversity simulation speed human-in-the-loop VR interface Social Navigation Social Rearrangement learned baselines heuristic baselines emergent behaviors