MaskedMimic: Unified Physics-Based Character Control Through Masked
  Motion Inpainting
Crafting a single, versatile physics-based controller that can breathe life
into interactive characters across a wide spectrum of scenarios represents an
exciting frontier in character animation. An ideal controller should support
diverse control modalities, such as sparse target keyframes, text instructions,
and scene information. While previous works have proposed physically simulated,
scene-aware control models, these systems have predominantly focused on
developing controllers that each specializes in a narrow set of tasks and
control modalities. This work presents MaskedMimic, a novel approach that
formulates physics-based character control as a general motion inpainting
problem. Our key insight is to train a single unified model to synthesize
motions from partial (masked) motion descriptions, such as masked keyframes,
objects, text descriptions, or any combination thereof. This is achieved by
leveraging motion tracking data and designing a scalable training method that
can effectively utilize diverse motion descriptions to produce coherent
animations. Through this process, our approach learns a physics-based
controller that provides an intuitive control interface without requiring
tedious reward engineering for all behaviors of interest. The resulting
controller supports a wide range of control modalities and enables seamless
transitions between disparate tasks. By unifying character control through
motion inpainting, MaskedMimic creates versatile virtual characters. These
characters can dynamically adapt to complex scenes and compose diverse motions
on demand, enabling more interactive and immersive experiences.
66f239eb50a4aeef226d2733
Chen Tessler
66f239eb50a4aeef226d2734
Yunrong Guo
66f239eb50a4aeef226d2735
Ofir Nabati
66f239eb50a4aeef226d2736
Gal Chechik
extracted_pending
2024-09-24T04:02:54.966Z
6493393f357b252af72196c5
https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
Gal Chechik
user
galchechik
66f239eb50a4aeef226d2737
Xue Bin Peng
MaskedMimic: Unified Physics-Based Character Control Through Masked
  Motion Inpainting
Crafting a single, versatile physics-based controller that can breathe life
into interactive characters across a wide spectrum of scenarios represents an
exciting frontier in character animation. An ideal controller should support
diverse control modalities, such as sparse target keyframes, text instructions,
and scene information. While previous works have proposed physically simulated,
scene-aware control models, these systems have predominantly focused on
developing controllers that each specializes in a narrow set of tasks and
control modalities. This work presents MaskedMimic, a novel approach that
formulates physics-based character control as a general motion inpainting
problem. Our key insight is to train a single unified model to synthesize
motions from partial (masked) motion descriptions, such as masked keyframes,
objects, text descriptions, or any combination thereof. This is achieved by
leveraging motion tracking data and designing a scalable training method that
can effectively utilize diverse motion descriptions to produce coherent
animations. Through this process, our approach learns a physics-based
controller that provides an intuitive control interface without requiring
tedious reward engineering for all behaviors of interest. The resulting
controller supports a wide range of control modalities and enables seamless
transitions between disparate tasks. By unifying character control through
motion inpainting, MaskedMimic creates versatile virtual characters. These
characters can dynamically adapt to complex scenes and compose diverse motions
on demand, enabling more interactive and immersive experiences.
physics-based controller control modalities sparse target keyframes text instructions scene-aware control models MaskedMimic general motion inpainting partial (masked) motion descriptions masked keyframes motion tracking data scalable training method coherent animations reward engineering intuitive control interface versatile virtual characters complex scenes diverse motions interactive experiences