Perpetual Humanoid Control for Real-time Simulated Avatars
We present a physics-based humanoid controller that achieves high-fidelity
motion imitation and fault-tolerant behavior in the presence of noisy input
(e.g. pose estimates from video or generated from language) and unexpected
falls. Our controller scales up to learning ten thousand motion clips without
using any external stabilizing forces and learns to naturally recover from
fail-state. Given reference motion, our controller can perpetually control
simulated avatars without requiring resets. At its core, we propose the
progressive multiplicative control policy (PMCP), which dynamically allocates
new network capacity to learn harder and harder motion sequences. PMCP allows
efficient scaling for learning from large-scale motion databases and adding new
tasks, such as fail-state recovery, without catastrophic forgetting. We
demonstrate the effectiveness of our controller by using it to imitate noisy
poses from video-based pose estimators and language-based motion generators in
a live and real-time multi-person avatar use case.
645dcb6403fc86c46b42313d
Zhengyi Luo
claimed_verified
2023-05-14T17:51:29.818Z
645eaf37f0f92653b9f2ba77
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VM-hXdknuRfeK3BD9FbyR.png
Luo
user
Zhengyiluo
645dcb6403fc86c46b42313e
Jinkun Cao
645dcb6403fc86c46b42313f
Alexander Winkler
645dcb6403fc86c46b423140
Kris Kitani
645dcb6403fc86c46b423141
Weipeng Xu
Perpetual Humanoid Control for Real-time Simulated Avatars
We present a physics-based humanoid controller that achieves high-fidelity
motion imitation and fault-tolerant behavior in the presence of noisy input
(e.g. pose estimates from video or generated from language) and unexpected
falls. Our controller scales up to learning ten thousand motion clips without
using any external stabilizing forces and learns to naturally recover from
fail-state. Given reference motion, our controller can perpetually control
simulated avatars without requiring resets. At its core, we propose the
progressive multiplicative control policy (PMCP), which dynamically allocates
new network capacity to learn harder and harder motion sequences. PMCP allows
efficient scaling for learning from large-scale motion databases and adding new
tasks, such as fail-state recovery, without catastrophic forgetting. We
demonstrate the effectiveness of our controller by using it to imitate noisy
poses from video-based pose estimators and language-based motion generators in
a live and real-time multi-person avatar use case.
A physics-based humanoid controller imitates motion and recovers from faults using a progressive multiplicative control policy, demonstrating high fidelity and scalability in noisy and dynamic environments.
progressive multiplicative control policy PMCP motion imitation fault-tolerant behavior noisy input pose estimates language-based motion generators simulated avatars fail-state recovery catastrophic forgetting