TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis
The gradual nature of a diffusion process that synthesizes samples in small
increments constitutes a key ingredient of Denoising Diffusion Probabilistic
Models (DDPM), which have presented unprecedented quality in image synthesis
and been recently explored in the motion domain. In this work, we propose to
adapt the gradual diffusion concept (operating along a diffusion time-axis)
into the temporal-axis of the motion sequence. Our key idea is to extend the
DDPM framework to support temporally varying denoising, thereby entangling the
two axes. Using our special formulation, we iteratively denoise a motion buffer
that contains a set of increasingly-noised poses, which auto-regressively
produces an arbitrarily long stream of frames. With a stationary diffusion
time-axis, in each diffusion step we increment only the temporal-axis of the
motion such that the framework produces a new, clean frame which is removed
from the beginning of the buffer, followed by a newly drawn noise vector that
is appended to it. This new mechanism paves the way towards a new framework for
long-term motion synthesis with applications to character animation and other
domains.
64c34ee97f9b6ae10e140367
Zihan Zhang
64c34ee97f9b6ae10e140368
Richard Liu
64c34ee97f9b6ae10e140369
Kfir Aberman
64c34ee97f9b6ae10e14036a
Rana Hanocka
admin_assigned
2023-07-28T09:36:12.018Z
630495ffce6b12280b193f30
/avatars/55b201131fa798c2a33d17dcab72fd1f.svg
Rana Hanocka
user
ranahanocka
TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis
The gradual nature of a diffusion process that synthesizes samples in small
increments constitutes a key ingredient of Denoising Diffusion Probabilistic
Models (DDPM), which have presented unprecedented quality in image synthesis
and been recently explored in the motion domain. In this work, we propose to
adapt the gradual diffusion concept (operating along a diffusion time-axis)
into the temporal-axis of the motion sequence. Our key idea is to extend the
DDPM framework to support temporally varying denoising, thereby entangling the
two axes. Using our special formulation, we iteratively denoise a motion buffer
that contains a set of increasingly-noised poses, which auto-regressively
produces an arbitrarily long stream of frames. With a stationary diffusion
time-axis, in each diffusion step we increment only the temporal-axis of the
motion such that the framework produces a new, clean frame which is removed
from the beginning of the buffer, followed by a newly drawn noise vector that
is appended to it. This new mechanism paves the way towards a new framework for
long-term motion synthesis with applications to character animation and other
domains.
The work adapts DDPM for motion synthesis by extending the diffusion framework to a temporal axis, enabling the generation of long streams of clean motion frames.
Denoising Diffusion Probabilistic Models DDPM diffusion process image synthesis motion domain temporal-axis temporally varying denoising motion buffer clean frame long-term motion synthesis character animation