BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike
  Animated Motion
We show, for the first time, that neural networks trained only on synthetic
data achieve state-of-the-art accuracy on the problem of 3D human pose and
shape (HPS) estimation from real images. Previous synthetic datasets have been
small, unrealistic, or lacked realistic clothing. Achieving sufficient realism
is non-trivial and we show how to do this for full bodies in motion.
Specifically, our BEDLAM dataset contains monocular RGB videos with
ground-truth 3D bodies in SMPL-X format. It includes a diversity of body
shapes, motions, skin tones, hair, and clothing. The clothing is realistically
simulated on the moving bodies using commercial clothing physics simulation. We
render varying numbers of people in realistic scenes with varied lighting and
camera motions. We then train various HPS regressors using BEDLAM and achieve
state-of-the-art accuracy on real-image benchmarks despite training with
synthetic data. We use BEDLAM to gain insights into what model design choices
are important for accuracy. With good synthetic training data, we find that a
basic method like HMR approaches the accuracy of the current SOTA method
(CLIFF). BEDLAM is useful for a variety of tasks and all images, ground truth
bodies, 3D clothing, support code, and more are available for research
purposes. Additionally, we provide detailed information about our synthetic
data generation pipeline, enabling others to generate their own datasets. See
the project page: https://bedlam.is.tue.mpg.de/.
649e5ff927145c4463174eb5
Michael J. Black
claimed_verified
2023-07-03T07:27:15.294Z
649fbababa7a05e3626546c8
/avatars/a9aa4b4914649527094fc1b12779e014.svg
Michael J. Black
user
MichaelJBlack
649e5ff927145c4463174eb6
Priyanka Patel
649e5ff927145c4463174eb7
Joachim Tesch
649e5ff927145c4463174eb8
Jinlong Yang
BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike
  Animated Motion
We show, for the first time, that neural networks trained only on synthetic
data achieve state-of-the-art accuracy on the problem of 3D human pose and
shape (HPS) estimation from real images. Previous synthetic datasets have been
small, unrealistic, or lacked realistic clothing. Achieving sufficient realism
is non-trivial and we show how to do this for full bodies in motion.
Specifically, our BEDLAM dataset contains monocular RGB videos with
ground-truth 3D bodies in SMPL-X format. It includes a diversity of body
shapes, motions, skin tones, hair, and clothing. The clothing is realistically
simulated on the moving bodies using commercial clothing physics simulation. We
render varying numbers of people in realistic scenes with varied lighting and
camera motions. We then train various HPS regressors using BEDLAM and achieve
state-of-the-art accuracy on real-image benchmarks despite training with
synthetic data. We use BEDLAM to gain insights into what model design choices
are important for accuracy. With good synthetic training data, we find that a
basic method like HMR approaches the accuracy of the current SOTA method
(CLIFF). BEDLAM is useful for a variety of tasks and all images, ground truth
bodies, 3D clothing, support code, and more are available for research
purposes. Additionally, we provide detailed information about our synthetic
data generation pipeline, enabling others to generate their own datasets. See
the project page: https://bedlam.is.tue.mpg.de/.
Synthetic data with realistic simulation of clothing dynamics and diverse scenarios achieves top accuracy in 3D human pose and shape estimation from real images.
neural networks 3D human pose and shape estimation synthetic data BEDLAM dataset RGB videos SMPL-X format body shapes motions skin tones hair clothing physics simulation real-image benchmarks HPS regressors HMR CLIFF