Emma-X: An Embodied Multimodal Action Model with Grounded Chain of
  Thought and Look-ahead Spatial Reasoning
Traditional reinforcement learning-based robotic control methods are often
task-specific and fail to generalize across diverse environments or unseen
objects and instructions. Visual Language Models (VLMs) demonstrate strong
scene understanding and planning capabilities but lack the ability to generate
actionable policies tailored to specific robotic embodiments. To address this,
Visual-Language-Action (VLA) models have emerged, yet they face challenges in
long-horizon spatial reasoning and grounded task planning. In this work, we
propose the Embodied Multimodal Action Model with Grounded Chain of Thought and
Look-ahead Spatial Reasoning, Emma-X. Emma-X leverages our constructed
hierarchical embodiment dataset based on BridgeV2, containing 60,000 robot
manipulation trajectories auto-annotated with grounded task reasoning and
spatial guidance. Additionally, we introduce a trajectory segmentation strategy
based on gripper states and motion trajectories, which can help mitigate
hallucination in grounding subtask reasoning generation. Experimental results
demonstrate that Emma-X achieves superior performance over competitive
baselines, particularly in real-world robotic tasks requiring spatial
reasoning.
676196e778b836c285b3bc20
Qi Sun
676196e778b836c285b3bc21
Pengfei Hong
admin_assigned
2024-12-18T13:35:07.709Z
62c3f44617c780a4f2ed603c
https://cdn-avatars.huggingface.co/v1/production/uploads/1657009213654-noauth.jpeg
Hong Pengfei
user
emrys-hong
676196e778b836c285b3bc22
Tej Deep Pala
claimed_verified
2025-05-28T10:13:58.384Z
633f875450d83f5065d459d6
/avatars/48312867ac2c05d589869972f90881b9.svg
Pala Tej Deep
user
Tej3
676196e778b836c285b3bc23
Vernon Toh
676196e778b836c285b3bc24
U-Xuan Tan
676196e778b836c285b3bc25
Deepanway Ghosal
admin_assigned
2024-12-18T13:34:34.419Z
62eced5e7e89e8d34df1a1ea
/avatars/99c9d8ba7e7722b7524d5d687cf96a25.svg
Deepanway Ghosal
user
dghosal
676196e778b836c285b3bc26
Soujanya Poria
admin_assigned
2024-12-18T13:34:27.305Z
626b626405fe1cb65725aca1
https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/ZVSbhynzpQhVGq9kGywW6.png
Soujanya Poria
user
soujanyaporia
Emma-X: An Embodied Multimodal Action Model with Grounded Chain of
  Thought and Look-ahead Spatial Reasoning
Traditional reinforcement learning-based robotic control methods are often
task-specific and fail to generalize across diverse environments or unseen
objects and instructions. Visual Language Models (VLMs) demonstrate strong
scene understanding and planning capabilities but lack the ability to generate
actionable policies tailored to specific robotic embodiments. To address this,
Visual-Language-Action (VLA) models have emerged, yet they face challenges in
long-horizon spatial reasoning and grounded task planning. In this work, we
propose the Embodied Multimodal Action Model with Grounded Chain of Thought and
Look-ahead Spatial Reasoning, Emma-X. Emma-X leverages our constructed
hierarchical embodiment dataset based on BridgeV2, containing 60,000 robot
manipulation trajectories auto-annotated with grounded task reasoning and
spatial guidance. Additionally, we introduce a trajectory segmentation strategy
based on gripper states and motion trajectories, which can help mitigate
hallucination in grounding subtask reasoning generation. Experimental results
demonstrate that Emma-X achieves superior performance over competitive
baselines, particularly in real-world robotic tasks requiring spatial
reasoning.
Emma-X, a multimodal action model with grounded chain of thought and look-ahead spatial reasoning, improves long-horizon spatial reasoning and grounded task planning in robot manipulation using a hierarchical embodiment dataset and trajectory segmentation strategy.
Visual-Language-Action models hierarchical embodiment dataset BridgeV2 trajectory segmentation grippler states motion trajectories grounded task reasoning spatial guidance long-horizon spatial reasoning grounded task planning