Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human
  Motion Generation
In the field of text-to-motion generation, Bert-type Masked Models (MoMask,
MMM) currently produce higher-quality outputs compared to GPT-type
autoregressive models (T2M-GPT). However, these Bert-type models often lack the
streaming output capability required for applications in video game and
multimedia environments, a feature inherent to GPT-type models. Additionally,
they demonstrate weaker performance in out-of-distribution generation. To
surpass the quality of BERT-type models while leveraging a GPT-type structure,
without adding extra refinement models that complicate scaling data, we propose
a novel architecture, Mogo (Motion Only Generate Once), which generates
high-quality lifelike 3D human motions by training a single transformer model.
Mogo consists of only two main components: 1) RVQ-VAE, a hierarchical residual
vector quantization variational autoencoder, which discretizes continuous
motion sequences with high precision; 2) Hierarchical Causal Transformer,
responsible for generating the base motion sequences in an autoregressive
manner while simultaneously inferring residuals across different layers.
Experimental results demonstrate that Mogo can generate continuous and cyclic
motion sequences up to 260 frames (13 seconds), surpassing the 196 frames (10
seconds) length limitation of existing datasets like HumanML3D. On the
HumanML3D test set, Mogo achieves a FID score of 0.079, outperforming both the
GPT-type model T2M-GPT (FID = 0.116), AttT2M (FID = 0.112) and the BERT-type
model MMM (FID = 0.080). Furthermore, our model achieves the best quantitative
performance in out-of-distribution generation.
675a5805e32fafc3f302070a
Dongjie Fu
extracted_pending
2024-12-12T03:27:03.907Z
647477a1e0b188d3cb20a910
/avatars/5da1cf4f7d727afc9320ed162bd38404.svg
Fragile Merak
user
Frag1le
Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human
  Motion Generation
In the field of text-to-motion generation, Bert-type Masked Models (MoMask,
MMM) currently produce higher-quality outputs compared to GPT-type
autoregressive models (T2M-GPT). However, these Bert-type models often lack the
streaming output capability required for applications in video game and
multimedia environments, a feature inherent to GPT-type models. Additionally,
they demonstrate weaker performance in out-of-distribution generation. To
surpass the quality of BERT-type models while leveraging a GPT-type structure,
without adding extra refinement models that complicate scaling data, we propose
a novel architecture, Mogo (Motion Only Generate Once), which generates
high-quality lifelike 3D human motions by training a single transformer model.
Mogo consists of only two main components: 1) RVQ-VAE, a hierarchical residual
vector quantization variational autoencoder, which discretizes continuous
motion sequences with high precision; 2) Hierarchical Causal Transformer,
responsible for generating the base motion sequences in an autoregressive
manner while simultaneously inferring residuals across different layers.
Experimental results demonstrate that Mogo can generate continuous and cyclic
motion sequences up to 260 frames (13 seconds), surpassing the 196 frames (10
seconds) length limitation of existing datasets like HumanML3D. On the
HumanML3D test set, Mogo achieves a FID score of 0.079, outperforming both the
GPT-type model T2M-GPT (FID = 0.116), AttT2M (FID = 0.112) and the BERT-type
model MMM (FID = 0.080). Furthermore, our model achieves the best quantitative
performance in out-of-distribution generation.
The proposed Mogo architecture, combining RVQ-VAE and Hierarchical Causal Transformer, generates high-quality, long-duration 3D human motions and achieves superior performance compared to existing BERT-type and GPT-type models, especially in out-of-distribution scenarios.
RVQ-VAE Hierarchical Causal Transformer FID score HumanML3D