Controllable Music Production with Diffusion Models and Guidance
  Gradients
We demonstrate how conditional generation from diffusion models can be used
to tackle a variety of realistic tasks in the production of music in 44.1kHz
stereo audio with sampling-time guidance. The scenarios we consider include
continuation, inpainting and regeneration of musical audio, the creation of
smooth transitions between two different music tracks, and the transfer of
desired stylistic characteristics to existing audio clips. We achieve this by
applying guidance at sampling time in a simple framework that supports both
reconstruction and classification losses, or any combination of the two. This
approach ensures that generated audio can match its surrounding context, or
conform to a class distribution or latent representation specified relative to
any suitable pre-trained classifier or embedding model.
65430e2b4539d4807059985b
Mark Levy
claimed_verified
2023-11-02T13:38:53.856Z
6543bd237a23df0e635d834c
/avatars/d2f6869ab09d13d8f4628a73bcbbd023.svg
Mark Levy
user
gamboviol
65430e2b4539d4807059985c
Bruno Di Giorgi
65430e2b4539d4807059985d
Floris Weers
claimed_verified
2024-03-15T11:02:55.040Z
65f41f1d6724ea19d7bc9052
/avatars/3171a092b40da7518bce00b80f7c6ff0.svg
Floris Weers
user
fweers
65430e2b4539d4807059985e
Angelos Katharopoulos
65430e2b4539d4807059985f
Tom Nickson
admin_assigned
2023-11-02T13:39:14.481Z
62cdd89b3529c21a2285baca
/avatars/1468fb8715bba441408b47ba834909dd.svg
Tom Nickson
user
TomNickson
Controllable Music Production with Diffusion Models and Guidance
  Gradients
We demonstrate how conditional generation from diffusion models can be used
to tackle a variety of realistic tasks in the production of music in 44.1kHz
stereo audio with sampling-time guidance. The scenarios we consider include
continuation, inpainting and regeneration of musical audio, the creation of
smooth transitions between two different music tracks, and the transfer of
desired stylistic characteristics to existing audio clips. We achieve this by
applying guidance at sampling time in a simple framework that supports both
reconstruction and classification losses, or any combination of the two. This
approach ensures that generated audio can match its surrounding context, or
conform to a class distribution or latent representation specified relative to
any suitable pre-trained classifier or embedding model.
Conditional generation from diffusion models is used for various music production tasks using sampling-time guidance, ensuring context matching and conforming to class distributions.
diffusion models conditional generation musical audio sampling-time guidance continuation inpainting regeneration smooth transitions stylistic characteristics reconstruction losses classification losses pre-trained classifier latent representation embedding model