WHAC: World-grounded Humans and Cameras
Estimating human and camera trajectories with accurate scale in the world
coordinate system from a monocular video is a highly desirable yet challenging
and ill-posed problem. In this study, we aim to recover expressive parametric
human models (i.e., SMPL-X) and corresponding camera poses jointly, by
leveraging the synergy between three critical players: the world, the human,
and the camera. Our approach is founded on two key observations. Firstly,
camera-frame SMPL-X estimation methods readily recover absolute human depth.
Secondly, human motions inherently provide absolute spatial cues. By
integrating these insights, we introduce a novel framework, referred to as
WHAC, to facilitate world-grounded expressive human pose and shape estimation
(EHPS) alongside camera pose estimation, without relying on traditional
optimization techniques. Additionally, we present a new synthetic dataset,
WHAC-A-Mole, which includes accurately annotated humans and cameras, and
features diverse interactive human motions as well as realistic camera
trajectories. Extensive experiments on both standard and newly established
benchmarks highlight the superiority and efficacy of our framework. We will
make the code and dataset publicly available.
67bc67e17727595ca5b7ddb4
Wanqi Yin
claimed_verified
2025-02-27T09:17:51.355Z
668f51fb8e8d87dbdd23caa9
/avatars/0b17cb0f3ad2c729f185cdccdad94e48.svg
Yin
user
waanqii
67bc67e17727595ca5b7ddb5
Zhongang Cai
admin_assigned
2025-02-24T16:14:30.375Z
652d06833b5997ed71ce5c46
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/xZTXEcnEogEmBm_ledJQr.jpeg
Zhongang Cai
user
caizhongang
67bc67e17727595ca5b7ddb6
Ruisi Wang
admin_assigned
2025-02-24T16:14:42.508Z
65b74305e602b6c2c9125480
/avatars/d36909e0f245bfeb632a4afc9d3fceca.svg
wang ruisi
user
wruisi
67bc67e17727595ca5b7ddb7
Fanzhou Wang
admin_assigned
2025-02-24T16:14:48.396Z
66ac94b7ccd0aed70992a8be
/avatars/98db346b7bfcb2040d4b58727a73d18b.svg
Fanzhou Wang
user
wentww
67bc67e17727595ca5b7ddb8
Chen Wei
67bc67e17727595ca5b7ddb9
Haiyi Mei
admin_assigned
2025-02-24T16:14:59.208Z
635a895ef4a106ecd9203b2d
/avatars/0ed9967f559582c2d93b5471b39f731a.svg
haiyimei
user
haiyimei
67bc67e17727595ca5b7ddba
Weiye Xiao
67bc67e17727595ca5b7ddbb
Zhitao Yang
67bc67e17727595ca5b7ddbc
Qingping Sun
admin_assigned
2025-02-24T16:15:19.079Z
65e20f88bb9c052e4171b857
/avatars/5dcb3fe293c53e051842baf9024d589b.svg
Qingping SUN
user
ttxskk
67bc67e17727595ca5b7ddbd
Atsushi Yamashita
admin_assigned
2025-02-24T16:15:25.136Z
6485d56c77076d551d4adedc
/avatars/0154474a6c6ec39a08f354b5dd69e1e3.svg
atsushi yamashita
user
atsu-yama
67bc67e17727595ca5b7ddbe
Ziwei Liu
admin_assigned
2025-02-24T16:14:15.911Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
67bc67e17727595ca5b7ddbf
Lei Yang
admin_assigned
2025-03-10T08:11:07.737Z
6626a471430a124253f197c8
/avatars/f5747fdbe495d1296fed9d16d8c95857.svg
yl-1993
user
yl-1993
WHAC: World-grounded Humans and Cameras
Estimating human and camera trajectories with accurate scale in the world
coordinate system from a monocular video is a highly desirable yet challenging
and ill-posed problem. In this study, we aim to recover expressive parametric
human models (i.e., SMPL-X) and corresponding camera poses jointly, by
leveraging the synergy between three critical players: the world, the human,
and the camera. Our approach is founded on two key observations. Firstly,
camera-frame SMPL-X estimation methods readily recover absolute human depth.
Secondly, human motions inherently provide absolute spatial cues. By
integrating these insights, we introduce a novel framework, referred to as
WHAC, to facilitate world-grounded expressive human pose and shape estimation
(EHPS) alongside camera pose estimation, without relying on traditional
optimization techniques. Additionally, we present a new synthetic dataset,
WHAC-A-Mole, which includes accurately annotated humans and cameras, and
features diverse interactive human motions as well as realistic camera
trajectories. Extensive experiments on both standard and newly established
benchmarks highlight the superiority and efficacy of our framework. We will
make the code and dataset publicly available.
A new framework, WHAC, jointly estimates expressive human models and camera poses from monocular video using synergies between human motion and camera perspectives without traditional optimization.
SMPL-X expressive human models EHPS camera poses WHAC WHAC-A-Mole synthetic dataset monocular video human motion camera trajectories