MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models
Recent advances in text-to-music generation models have opened new avenues in
musical creativity. However, music generation usually involves iterative
refinements, and how to edit the generated music remains a significant
challenge. This paper introduces a novel approach to the editing of music
generated by such models, enabling the modification of specific attributes,
such as genre, mood and instrument, while maintaining other aspects unchanged.
Our method transforms text editing to latent space manipulation while
adding an extra constraint to enforce consistency. It seamlessly integrates
with existing pretrained text-to-music diffusion models without requiring
additional training. Experimental results demonstrate superior performance over
both zero-shot and certain supervised baselines in style and timbre transfer
evaluations. Additionally, we showcase the practical applicability of our
approach in real-world music editing scenarios.
65c98dfa8f6973864f79a1af
Yixiao Zhang
admin_assigned
2024-02-12T16:26:59.211Z
6376a5aad49217e154f580aa
https://cdn-avatars.huggingface.co/v1/production/uploads/1668720030380-noauth.jpeg
Yixiao ZHang
user
ldzhangyx
65c98dfa8f6973864f79a1b0
Yukara Ikemiya
admin_assigned
2024-02-12T16:27:06.047Z
62fbbdcd21c444a56f7a6159
/avatars/3f83effd52773f0806f58331d8734861.svg
Yukara Ikemiya
user
yukara13
65c98dfa8f6973864f79a1b1
Gus Xia
65c98dfa8f6973864f79a1b2
Naoki Murata
admin_assigned
2024-02-12T16:27:21.604Z
63e1e9627fbb6ae4d4f59d76
/avatars/22197a8c85232135609331766299c3e7.svg
Naoki Murata
user
muttyjp
65c98dfa8f6973864f79a1b3
Marco Martínez
65c98dfa8f6973864f79a1b4
Wei-Hsiang Liao
65c98dfa8f6973864f79a1b5
Yuki Mitsufuji
65c98dfa8f6973864f79a1b6
Simon Dixon
admin_assigned
2024-02-12T16:28:00.696Z
643ac28aff50448bcfc24b19
/avatars/18d201b94829bae9985dbb4ec618e1ff.svg
Simon Dixon
user
Dexdixon
MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models
Recent advances in text-to-music generation models have opened new avenues in
musical creativity. However, music generation usually involves iterative
refinements, and how to edit the generated music remains a significant
challenge. This paper introduces a novel approach to the editing of music
generated by such models, enabling the modification of specific attributes,
such as genre, mood and instrument, while maintaining other aspects unchanged.
Our method transforms text editing to latent space manipulation while
adding an extra constraint to enforce consistency. It seamlessly integrates
with existing pretrained text-to-music diffusion models without requiring
additional training. Experimental results demonstrate superior performance over
both zero-shot and certain supervised baselines in style and timbre transfer
evaluations. Additionally, we showcase the practical applicability of our
approach in real-world music editing scenarios.
Latent space manipulation allows for the editing of music generated from text, enabling attribute-specific changes while maintaining consistency, demonstrated superior performance in style and timbre transfer, and is applicable in real-world scenarios.
latent space manipulation diffusion models text-to-music generation genre mood instrument style transfer timbre transfer