Make-An-Animation: Large-Scale Text-conditional 3D Human Motion
  Generation
Text-guided human motion generation has drawn significant interest because of
its impactful applications spanning animation and robotics. Recently,
application of diffusion models for motion generation has enabled improvements
in the quality of generated motions. However, existing approaches are limited
by their reliance on relatively small-scale motion capture data, leading to
poor performance on more diverse, in-the-wild prompts. In this paper, we
introduce Make-An-Animation, a text-conditioned human motion generation model
which learns more diverse poses and prompts from large-scale image-text
datasets, enabling significant improvement in performance over prior works.
Make-An-Animation is trained in two stages. First, we train on a curated
large-scale dataset of (text, static pseudo-pose) pairs extracted from
image-text datasets. Second, we fine-tune on motion capture data, adding
additional layers to model the temporal dimension. Unlike prior diffusion
models for motion generation, Make-An-Animation uses a U-Net architecture
similar to recent text-to-video generation models. Human evaluation of motion
realism and alignment with input text shows that our model reaches
state-of-the-art performance on text-to-motion generation.
646421964bf912292228e118
Samaneh Azadi
646421964bf912292228e119
Akbar Shah
646421964bf912292228e11a
Thomas Hayes
646421964bf912292228e11b
Devi Parikh
admin_assigned
2023-05-26T09:34:36.752Z
6340753995c20b9447379c67
/avatars/f793c00a98335cd8f7efce41d403da16.svg
Devi Parikh
user
deviparikh-genai
646421964bf912292228e11c
Sonal Gupta
admin_assigned
2023-05-26T09:35:08.245Z
634ecb814677a5891c0c7026
/avatars/a20cf325e4fe0a227ff80694ffea4a3d.svg
Sonal Gupta
user
thenextgenai
Make-An-Animation: Large-Scale Text-conditional 3D Human Motion
  Generation
Text-guided human motion generation has drawn significant interest because of
its impactful applications spanning animation and robotics. Recently,
application of diffusion models for motion generation has enabled improvements
in the quality of generated motions. However, existing approaches are limited
by their reliance on relatively small-scale motion capture data, leading to
poor performance on more diverse, in-the-wild prompts. In this paper, we
introduce Make-An-Animation, a text-conditioned human motion generation model
which learns more diverse poses and prompts from large-scale image-text
datasets, enabling significant improvement in performance over prior works.
Make-An-Animation is trained in two stages. First, we train on a curated
large-scale dataset of (text, static pseudo-pose) pairs extracted from
image-text datasets. Second, we fine-tune on motion capture data, adding
additional layers to model the temporal dimension. Unlike prior diffusion
models for motion generation, Make-An-Animation uses a U-Net architecture
similar to recent text-to-video generation models. Human evaluation of motion
realism and alignment with input text shows that our model reaches
state-of-the-art performance on text-to-motion generation.
A text-conditioned human motion generation model, Make-An-Animation, achieves state-of-the-art performance by learning from large-scale image-text datasets and using a U-Net architecture for both spatial and temporal modeling.
diffusion models Make-An-Animation text-conditioned human motion generation U-Net architecture large-scale image-text datasets temporal dimension text-to-motion generation