MotionStreamer: Streaming Motion Generation via Diffusion-based
  Autoregressive Model in Causal Latent Space
This paper addresses the challenge of text-conditioned streaming motion
generation, which requires us to predict the next-step human pose based on
variable-length historical motions and incoming texts. Existing methods
struggle to achieve streaming motion generation, e.g., diffusion models are
constrained by pre-defined motion lengths, while GPT-based methods suffer from
delayed response and error accumulation problem due to discretized non-causal
tokenization. To solve these problems, we propose MotionStreamer, a novel
framework that incorporates a continuous causal latent space into a
probabilistic autoregressive model. The continuous latents mitigate information
loss caused by discretization and effectively reduce error accumulation during
long-term autoregressive generation. In addition, by establishing temporal
causal dependencies between current and historical motion latents, our model
fully utilizes the available information to achieve accurate online motion
decoding. Experiments show that our method outperforms existing approaches
while offering more applications, including multi-round generation, long-term
generation, and dynamic motion composition. Project Page:
https://zju3dv.github.io/MotionStreamer/
67dd03fb2672aa3643252e8c
Lixing Xiao
claimed_verified
2025-03-21T11:40:07.470Z
65220fedc709aaca9aa63061
/avatars/36e56199fa8c98ff3430636567a0ed62.svg
Lixing Xiao
user
lxxiao
67dd03fb2672aa3643252e8d
Shunlin Lu
67dd03fb2672aa3643252e8e
Huaijin Pi
claimed_verified
2025-06-02T07:52:07.238Z
6836adaf8ec432fdc7e1c550
/avatars/ca21e0022e43fae7eddcd4b58aa7406f.svg
Huaijin Pi
user
huaijinpi
67dd03fb2672aa3643252e8f
Ke Fan
67dd03fb2672aa3643252e90
Liang Pan
claimed_verified
2025-04-01T15:59:42.952Z
6630cc7e9ee8861dd0b9bdbd
/avatars/01da7d60cbb107d58329bbb80d924eb4.svg
Liang Pan
user
lianganimation
67dd03fb2672aa3643252e91
Yueer Zhou
67dd03fb2672aa3643252e92
Ziyong Feng
67dd03fb2672aa3643252e93
Xiaowei Zhou
67dd03fb2672aa3643252e94
Sida Peng
67dd03fb2672aa3643252e95
Jingbo Wang
MotionStreamer: Streaming Motion Generation via Diffusion-based
  Autoregressive Model in Causal Latent Space
This paper addresses the challenge of text-conditioned streaming motion
generation, which requires us to predict the next-step human pose based on
variable-length historical motions and incoming texts. Existing methods
struggle to achieve streaming motion generation, e.g., diffusion models are
constrained by pre-defined motion lengths, while GPT-based methods suffer from
delayed response and error accumulation problem due to discretized non-causal
tokenization. To solve these problems, we propose MotionStreamer, a novel
framework that incorporates a continuous causal latent space into a
probabilistic autoregressive model. The continuous latents mitigate information
loss caused by discretization and effectively reduce error accumulation during
long-term autoregressive generation. In addition, by establishing temporal
causal dependencies between current and historical motion latents, our model
fully utilizes the available information to achieve accurate online motion
decoding. Experiments show that our method outperforms existing approaches
while offering more applications, including multi-round generation, long-term
generation, and dynamic motion composition. Project Page:
https://zju3dv.github.io/MotionStreamer/
MotionStreamer framework enhances text-conditioned streaming motion generation by using continuous causal latents in a probabilistic autoregressive model, addressing issues of error accumulation and delayed response.
diffusion models GPT-based methods streaming motion generation continuous causal latent space probabilistic autoregressive model error accumulation online motion decoding