InsActor: Instruction-driven Physics-based Characters
Generating animation of physics-based characters with intuitive control has
long been a desirable task with numerous applications. However, generating
physically simulated animations that reflect high-level human instructions
remains a difficult problem due to the complexity of physical environments and
the richness of human language. In this paper, we present InsActor, a
principled generative framework that leverages recent advancements in
diffusion-based human motion models to produce instruction-driven animations of
physics-based characters. Our framework empowers InsActor to capture complex
relationships between high-level human instructions and character motions by
employing diffusion policies for flexibly conditioned motion planning. To
overcome invalid states and infeasible state transitions in planned motions,
InsActor discovers low-level skills and maps plans to latent skill sequences in
a compact latent space. Extensive experiments demonstrate that InsActor
achieves state-of-the-art results on various tasks, including
instruction-driven motion generation and instruction-driven waypoint heading.
Notably, the ability of InsActor to generate physically simulated animations
using high-level human instructions makes it a valuable tool, particularly in
executing long-horizon tasks with a rich set of instructions.
658e50bf4a24a387789af5e4
Jiawei Ren
claimed_verified
2023-12-29T09:00:19.773Z
623f533a28672458f749b8e9
https://cdn-avatars.huggingface.co/v1/production/uploads/1651044799068-623f533a28672458f749b8e9.png
Jiawe Ren
user
jiawei011
658e50bf4a24a387789af5e5
Mingyuan Zhang
658e50bf4a24a387789af5e6
Cunjun Yu
658e50bf4a24a387789af5e7
Xiao Ma
claimed_verified
2025-07-24T09:15:37.691Z
6478597d91398856110a6738
/avatars/c3bc61eb7554ac21946b424e1314f1a7.svg
Xiao Ma
user
yusufma555
658e50bf4a24a387789af5e8
Liang Pan
claimed_verified
2024-11-05T16:13:01.142Z
64ef30fb0de94d31b7921175
/avatars/d4f4c7be460befc1cedec13bbf9db972.svg
pan
user
pldeqiushui
658e50bf4a24a387789af5e9
Ziwei Liu
InsActor: Instruction-driven Physics-based Characters
Generating animation of physics-based characters with intuitive control has
long been a desirable task with numerous applications. However, generating
physically simulated animations that reflect high-level human instructions
remains a difficult problem due to the complexity of physical environments and
the richness of human language. In this paper, we present InsActor, a
principled generative framework that leverages recent advancements in
diffusion-based human motion models to produce instruction-driven animations of
physics-based characters. Our framework empowers InsActor to capture complex
relationships between high-level human instructions and character motions by
employing diffusion policies for flexibly conditioned motion planning. To
overcome invalid states and infeasible state transitions in planned motions,
InsActor discovers low-level skills and maps plans to latent skill sequences in
a compact latent space. Extensive experiments demonstrate that InsActor
achieves state-of-the-art results on various tasks, including
instruction-driven motion generation and instruction-driven waypoint heading.
Notably, the ability of InsActor to generate physically simulated animations
using high-level human instructions makes it a valuable tool, particularly in
executing long-horizon tasks with a rich set of instructions.
InsActor, a generative framework using diffusion-based human motion models, produces high-quality physics-based animations guided by human instructions through diffusion policies and compact latent skill sequences.
diffusion-based human motion models diffusion policies latent skill sequences