Motion-Aware Concept Alignment for Consistent Video Editing
We introduce MoCA-Video (Motion-Aware Concept Alignment in Video), a
training-free framework bridging the gap between image-domain semantic mixing
and video. Given a generated video and a user-provided reference image,
MoCA-Video injects the semantic features of the reference image into a specific
object within the video, while preserving the original motion and visual
context. Our approach leverages a diagonal denoising schedule and
class-agnostic segmentation to detect and track objects in the latent space and
precisely control the spatial location of the blended objects. To ensure
temporal coherence, we incorporate momentum-based semantic corrections and
gamma residual noise stabilization for smooth frame transitions. We evaluate
MoCA's performance using the standard SSIM, image-level LPIPS, temporal LPIPS,
and introduce a novel metric CASS (Conceptual Alignment Shift Score) to
evaluate the consistency and effectiveness of the visual shifts between the
source prompt and the modified video frames. Using self-constructed dataset,
MoCA-Video outperforms current baselines, achieving superior spatial
consistency, coherent motion, and a significantly higher CASS score, despite
having no training or fine-tuning. MoCA-Video demonstrates that structured
manipulation in the diffusion noise trajectory allows for controllable,
high-quality video synthesis.
68403832646314cfc20a5de2
Tong Zhang
claimed_verified
2025-06-04T15:02:54.641Z
65c898197faf326d067e2c0d
/avatars/dde3c047e627bacd69e0f063a4a833b2.svg
Tong Zhang
user
Tong98Zhang
68403832646314cfc20a5de3
Juan C Leon Alcazar
68403832646314cfc20a5de4
Bernard Ghanem
Motion-Aware Concept Alignment for Consistent Video Editing
We introduce MoCA-Video (Motion-Aware Concept Alignment in Video), a
training-free framework bridging the gap between image-domain semantic mixing
and video. Given a generated video and a user-provided reference image,
MoCA-Video injects the semantic features of the reference image into a specific
object within the video, while preserving the original motion and visual
context. Our approach leverages a diagonal denoising schedule and
class-agnostic segmentation to detect and track objects in the latent space and
precisely control the spatial location of the blended objects. To ensure
temporal coherence, we incorporate momentum-based semantic corrections and
gamma residual noise stabilization for smooth frame transitions. We evaluate
MoCA's performance using the standard SSIM, image-level LPIPS, temporal LPIPS,
and introduce a novel metric CASS (Conceptual Alignment Shift Score) to
evaluate the consistency and effectiveness of the visual shifts between the
source prompt and the modified video frames. Using self-constructed dataset,
MoCA-Video outperforms current baselines, achieving superior spatial
consistency, coherent motion, and a significantly higher CASS score, despite
having no training or fine-tuning. MoCA-Video demonstrates that structured
manipulation in the diffusion noise trajectory allows for controllable,
high-quality video synthesis.
MoCA-Video injects semantic features from a reference image into a video object, preserving motion and visual context, and outperforms baselines using novel metrics.
MoCA-Video Motion-Aware Concept Alignment image-domain semantic mixing diagonal denoising schedule class-agnostic segmentation latent space momentum-based semantic corrections gamma residual noise stabilization SSIM image-level LPIPS temporal LPIPS CASS Conceptual Alignment Shift Score diffusion noise trajectory video synthesis structured manipulation