UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction,
  Forecasting, and Generation
Egocentric human motion generation and forecasting with scene-context is
crucial for enhancing AR/VR experiences, improving human-robot interaction,
advancing assistive technologies, and enabling adaptive healthcare solutions by
accurately predicting and simulating movement from a first-person perspective.
However, existing methods primarily focus on third-person motion synthesis with
structured 3D scene contexts, limiting their effectiveness in real-world
egocentric settings where limited field of view, frequent occlusions, and
dynamic cameras hinder scene perception. To bridge this gap, we introduce
Egocentric Motion Generation and Egocentric Motion Forecasting, two novel tasks
that utilize first-person images for scene-aware motion synthesis without
relying on explicit 3D scene. We propose UniEgoMotion, a unified conditional
motion diffusion model with a novel head-centric motion representation tailored
for egocentric devices. UniEgoMotion's simple yet effective design supports
egocentric motion reconstruction, forecasting, and generation from first-person
visual inputs in a unified framework. Unlike previous works that overlook scene
semantics, our model effectively extracts image-based scene context to infer
plausible 3D motion. To facilitate training, we introduce EE4D-Motion, a
large-scale dataset derived from EgoExo4D, augmented with pseudo-ground-truth
3D motion annotations. UniEgoMotion achieves state-of-the-art performance in
egocentric motion reconstruction and is the first to generate motion from a
single egocentric image. Extensive evaluations demonstrate the effectiveness of
our unified framework, setting a new benchmark for egocentric motion modeling
and unlocking new possibilities for egocentric applications.
68938b44741a16f544fbcdaf
Chaitanya Patel
claimed_verified
2025-08-06T19:12:20.645Z
66f10fa5f9cf1bf7e6798d15
/avatars/ff416c6778091eaf13f6c233a5e42244.svg
Chaitanya Patel
user
chaitanya100100
68938b44741a16f544fbcdb0
Hiroki Nakamura
68938b44741a16f544fbcdb1
Yuta Kyuragi
68938b44741a16f544fbcdb2
Kazuki Kozuka
68938b44741a16f544fbcdb3
Juan Carlos Niebles
68938b44741a16f544fbcdb4
Ehsan Adeli
UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction,
  Forecasting, and Generation
Egocentric human motion generation and forecasting with scene-context is
crucial for enhancing AR/VR experiences, improving human-robot interaction,
advancing assistive technologies, and enabling adaptive healthcare solutions by
accurately predicting and simulating movement from a first-person perspective.
However, existing methods primarily focus on third-person motion synthesis with
structured 3D scene contexts, limiting their effectiveness in real-world
egocentric settings where limited field of view, frequent occlusions, and
dynamic cameras hinder scene perception. To bridge this gap, we introduce
Egocentric Motion Generation and Egocentric Motion Forecasting, two novel tasks
that utilize first-person images for scene-aware motion synthesis without
relying on explicit 3D scene. We propose UniEgoMotion, a unified conditional
motion diffusion model with a novel head-centric motion representation tailored
for egocentric devices. UniEgoMotion's simple yet effective design supports
egocentric motion reconstruction, forecasting, and generation from first-person
visual inputs in a unified framework. Unlike previous works that overlook scene
semantics, our model effectively extracts image-based scene context to infer
plausible 3D motion. To facilitate training, we introduce EE4D-Motion, a
large-scale dataset derived from EgoExo4D, augmented with pseudo-ground-truth
3D motion annotations. UniEgoMotion achieves state-of-the-art performance in
egocentric motion reconstruction and is the first to generate motion from a
single egocentric image. Extensive evaluations demonstrate the effectiveness of
our unified framework, setting a new benchmark for egocentric motion modeling
and unlocking new possibilities for egocentric applications.
A unified conditional motion diffusion model, UniEgoMotion, is introduced for egocentric motion generation and forecasting using first-person images, achieving state-of-the-art performance and generating motion from a single image.
egocentric human motion generation egocentric motion forecasting scene-context AR/VR experiences human-robot interaction assistive technologies adaptive healthcare solutions first-person perspective third-person motion synthesis structured 3D scene contexts field of view occlusions dynamic cameras scene-aware motion synthesis head-centric motion representation egocentric devices unified framework image-based scene context plausible 3D motion EE4D-Motion EgoExo4D pseudo-ground-truth 3D motion annotations egocentric motion reconstruction unified conditional motion diffusion model