MoMask: Generative Masked Modeling of 3D Human Motions
We introduce MoMask, a novel masked modeling framework for text-driven 3D
human motion generation. In MoMask, a hierarchical quantization scheme is
employed to represent human motion as multi-layer discrete motion tokens with
high-fidelity details. Starting at the base layer, with a sequence of motion
tokens obtained by vector quantization, the residual tokens of increasing
orders are derived and stored at the subsequent layers of the hierarchy. This
is consequently followed by two distinct bidirectional transformers. For the
base-layer motion tokens, a Masked Transformer is designated to predict
randomly masked motion tokens conditioned on text input at training stage.
During generation (i.e. inference) stage, starting from an empty sequence, our
Masked Transformer iteratively fills up the missing tokens; Subsequently, a
Residual Transformer learns to progressively predict the next-layer tokens
based on the results from current layer. Extensive experiments demonstrate that
MoMask outperforms the state-of-art methods on the text-to-motion generation
task, with an FID of 0.045 (vs e.g. 0.141 of T2M-GPT) on the HumanML3D dataset,
and 0.228 (vs 0.514) on KIT-ML, respectively. MoMask can also be seamlessly
applied in related tasks without further model fine-tuning, such as text-guided
temporal inpainting.
656d3e0e4ab7bc884dbe6d47
Chuan Guo
admin_assigned
2023-12-04T20:51:33.037Z
656d77a08a37acfa3f275d46
https://cdn-avatars.huggingface.co/v1/production/uploads/656d77a08a37acfa3f275d46/3S_TakMfG5J8oZ6qDs-SX.jpeg
chuan guo
user
Ericguo5513
656d3e0e4ab7bc884dbe6d48
Yuxuan Mu
claimed_verified
2023-12-05T08:40:51.493Z
64ac2cce5769f5f88bc25475
https://cdn-avatars.huggingface.co/v1/production/uploads/64ac2cce5769f5f88bc25475/SMpNfbmDQd7IRIF0WBttV.jpeg
Mu
user
MeYourHint
656d3e0e4ab7bc884dbe6d49
Muhammad Gohar Javed
claimed_verified
2023-12-05T08:40:47.071Z
656e260e801ed9952ffee88b
/avatars/d9032cf3fed127d1234953af6b551fa8.svg
Muhammad Gohar Javed
user
javed4
656d3e0e4ab7bc884dbe6d4a
Sen Wang
claimed_verified
2023-12-05T08:40:43.595Z
64384682581e6bf0fa9ce6f8
/avatars/b4b5b4ed8b34b8d225f27c13d99985cd.svg
Sen Wang
user
wangsenxxz
656d3e0e4ab7bc884dbe6d4b
Li Cheng
MoMask: Generative Masked Modeling of 3D Human Motions
We introduce MoMask, a novel masked modeling framework for text-driven 3D
human motion generation. In MoMask, a hierarchical quantization scheme is
employed to represent human motion as multi-layer discrete motion tokens with
high-fidelity details. Starting at the base layer, with a sequence of motion
tokens obtained by vector quantization, the residual tokens of increasing
orders are derived and stored at the subsequent layers of the hierarchy. This
is consequently followed by two distinct bidirectional transformers. For the
base-layer motion tokens, a Masked Transformer is designated to predict
randomly masked motion tokens conditioned on text input at training stage.
During generation (i.e. inference) stage, starting from an empty sequence, our
Masked Transformer iteratively fills up the missing tokens; Subsequently, a
Residual Transformer learns to progressively predict the next-layer tokens
based on the results from current layer. Extensive experiments demonstrate that
MoMask outperforms the state-of-art methods on the text-to-motion generation
task, with an FID of 0.045 (vs e.g. 0.141 of T2M-GPT) on the HumanML3D dataset,
and 0.228 (vs 0.514) on KIT-ML, respectively. MoMask can also be seamlessly
applied in related tasks without further model fine-tuning, such as text-guided
temporal inpainting.
MoMask, a masked modeling framework using hierarchical quantization and bidirectional transformers, excels in text-to-motion generation and related tasks with high fidelity and competitive performance.
masked modeling hierarchical quantization motion tokens vector quantization residual tokens bidirectional transformers Masked Transformer Residual Transformer text-to-motion generation FID score HumanML3D KIT-ML temporal inpainting