MotionCLR: Motion Generation and Training-free Editing via Understanding
  Attention Mechanisms
This research delves into the problem of interactive editing of human motion
generation. Previous motion diffusion models lack explicit modeling of the
word-level text-motion correspondence and good explainability, hence
restricting their fine-grained editing ability. To address this issue, we
propose an attention-based motion diffusion model, namely MotionCLR, with CLeaR
modeling of attention mechanisms. Technically, MotionCLR models the in-modality
and cross-modality interactions with self-attention and cross-attention,
respectively. More specifically, the self-attention mechanism aims to measure
the sequential similarity between frames and impacts the order of motion
features. By contrast, the cross-attention mechanism works to find the
fine-grained word-sequence correspondence and activate the corresponding
timesteps in the motion sequence. Based on these key properties, we develop a
versatile set of simple yet effective motion editing methods via manipulating
attention maps, such as motion (de-)emphasizing, in-place motion replacement,
and example-based motion generation, etc. For further verification of the
explainability of the attention mechanism, we additionally explore the
potential of action-counting and grounded motion generation ability via
attention maps. Our experimental results show that our method enjoys good
generation and editing ability with good explainability.
671b2439bc2e7f71a43de086
Ling-Hao Chen
claimed_verified
2024-10-28T11:04:17.380Z
63109a4d61cab0446e48c83b
https://cdn-avatars.huggingface.co/v1/production/uploads/63109a4d61cab0446e48c83b/JQlVkQp0ok586ND1GmB0w.png
Ling-Hao Chen
user
EvanTHU
671b2439bc2e7f71a43de087
Wenxun Dai
671b2439bc2e7f71a43de088
Xuan Ju
671b2439bc2e7f71a43de089
Shunlin Lu
671b2439bc2e7f71a43de08a
Lei Zhang
MotionCLR: Motion Generation and Training-free Editing via Understanding
  Attention Mechanisms
This research delves into the problem of interactive editing of human motion
generation. Previous motion diffusion models lack explicit modeling of the
word-level text-motion correspondence and good explainability, hence
restricting their fine-grained editing ability. To address this issue, we
propose an attention-based motion diffusion model, namely MotionCLR, with CLeaR
modeling of attention mechanisms. Technically, MotionCLR models the in-modality
and cross-modality interactions with self-attention and cross-attention,
respectively. More specifically, the self-attention mechanism aims to measure
the sequential similarity between frames and impacts the order of motion
features. By contrast, the cross-attention mechanism works to find the
fine-grained word-sequence correspondence and activate the corresponding
timesteps in the motion sequence. Based on these key properties, we develop a
versatile set of simple yet effective motion editing methods via manipulating
attention maps, such as motion (de-)emphasizing, in-place motion replacement,
and example-based motion generation, etc. For further verification of the
explainability of the attention mechanism, we additionally explore the
potential of action-counting and grounded motion generation ability via
attention maps. Our experimental results show that our method enjoys good
generation and editing ability with good explainability.
MotionCLR, an attention-based motion diffusion model, enhances interactive motion editing by explicitly modeling word-level text-motion correspondence and improving explainability through self-attention and cross-attention mechanisms.
attention-based motion diffusion model MotionCLR CLeaR modeling self-attention cross-attention text-motion correspondence motion editing explainability attention maps motion (de-)emphasizing in-place motion replacement example-based motion generation action-counting grounded motion generation