STR-Match: Matching SpatioTemporal Relevance Score for Training-Free
  Video Editing
Previous text-guided video editing methods often suffer from temporal
inconsistency, motion distortion, and-most notably-limited domain
transformation. We attribute these limitations to insufficient modeling of
spatiotemporal pixel relevance during the editing process. To address this, we
propose STR-Match, a training-free video editing algorithm that produces
visually appealing and spatiotemporally coherent videos through latent
optimization guided by our novel STR score. The score captures spatiotemporal
pixel relevance across adjacent frames by leveraging 2D spatial attention and
1D temporal modules in text-to-video (T2V) diffusion models, without the
overhead of computationally expensive 3D attention mechanisms. Integrated into
a latent optimization framework with a latent mask, STR-Match generates
temporally consistent and visually faithful videos, maintaining strong
performance even under significant domain transformations while preserving key
visual attributes of the source. Extensive experiments demonstrate that
STR-Match consistently outperforms existing methods in both visual quality and
spatiotemporal consistency.
68637f0d588cea0da970c95e
Junsung Lee
claimed_verified
2025-07-03T07:32:55.680Z
6719de3235b6494469ab69f6
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/67l_hybjE-E2nH8_EyBuO.png
Junsung Lee
user
jslee525
68637f0d588cea0da970c95f
Junoh Kang
68637f0d588cea0da970c960
Bohyung Han
STR-Match: Matching SpatioTemporal Relevance Score for Training-Free
  Video Editing
Previous text-guided video editing methods often suffer from temporal
inconsistency, motion distortion, and-most notably-limited domain
transformation. We attribute these limitations to insufficient modeling of
spatiotemporal pixel relevance during the editing process. To address this, we
propose STR-Match, a training-free video editing algorithm that produces
visually appealing and spatiotemporally coherent videos through latent
optimization guided by our novel STR score. The score captures spatiotemporal
pixel relevance across adjacent frames by leveraging 2D spatial attention and
1D temporal modules in text-to-video (T2V) diffusion models, without the
overhead of computationally expensive 3D attention mechanisms. Integrated into
a latent optimization framework with a latent mask, STR-Match generates
temporally consistent and visually faithful videos, maintaining strong
performance even under significant domain transformations while preserving key
visual attributes of the source. Extensive experiments demonstrate that
STR-Match consistently outperforms existing methods in both visual quality and
spatiotemporal consistency.
STR-Match, a training-free video editing algorithm, enhances spatiotemporal coherence in videos using latent optimization guided by a novel STR score, which leverages 2D spatial and 1D temporal attention in T2V diffusion models.
STR-Match latent optimization STR score 2D spatial attention 1D temporal modules T2V diffusion models spatiotemporal coherence visual quality spatiotemporal consistency