Presto! Distilling Steps and Layers for Accelerating Music Generation
Despite advances in diffusion-based text-to-music (TTM) methods, efficient,
high-quality generation remains a challenge. We introduce Presto!, an approach
to inference acceleration for score-based diffusion transformers via reducing
both sampling steps and cost per step. To reduce steps, we develop a new
score-based distribution matching distillation (DMD) method for the EDM-family
of diffusion models, the first GAN-based distillation method for TTM. To reduce
the cost per step, we develop a simple, but powerful improvement to a recent
layer distillation method that improves learning via better preserving hidden
state variance. Finally, we combine our step and layer distillation methods
together for a dual-faceted approach. We evaluate our step and layer
distillation methods independently and show each yield best-in-class
performance. Our combined distillation method can generate high-quality outputs
with improved diversity, accelerating our base model by 10-18x (230/435ms
latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) --
the fastest high-quality TTM to our knowledge. Sound examples can be found at
https://presto-music.github.io/web/.
6704aababd2bdd04b6adf4ae
Zachary Novack
extracted_confirmed
2024-10-08T03:59:43.629Z
643060c6cb3fe707b24c53a2
https://cdn-avatars.huggingface.co/v1/production/uploads/643060c6cb3fe707b24c53a2/MIoM9hrX0vV4XRyrm-4Kz.jpeg
Zachary Novack
user
ZacharyNovack
6704aababd2bdd04b6adf4af
Ge Zhu
6704aababd2bdd04b6adf4b0
Jonah Casebeer
claimed_verified
2025-04-24T09:11:46.606Z
67575907a1d686e1662863b3
/avatars/d3480b64822b1fa68ce24659e8a5cc7f.svg
Jonah Casebeer
user
jmcasebeer
6704aababd2bdd04b6adf4b1
Julian McAuley
6704aababd2bdd04b6adf4b2
Taylor Berg-Kirkpatrick
admin_assigned
2024-10-08T09:29:04.746Z
66845caa4b898a41b7af775c
/avatars/e205290cd372b6b464509de279f6642d.svg
Taylor Berg-Kirkpatrick
user
tberg12
6704aababd2bdd04b6adf4b3
Nicholas J. Bryan
claimed_verified
2024-10-08T07:19:15.590Z
648119ee7a741c7f33f49f25
/avatars/183d2143ba20e1cc8712c63c055aadd7.svg
Nicholas J. Bryan
user
Njb
Presto! Distilling Steps and Layers for Accelerating Music Generation
Despite advances in diffusion-based text-to-music (TTM) methods, efficient,
high-quality generation remains a challenge. We introduce Presto!, an approach
to inference acceleration for score-based diffusion transformers via reducing
both sampling steps and cost per step. To reduce steps, we develop a new
score-based distribution matching distillation (DMD) method for the EDM-family
of diffusion models, the first GAN-based distillation method for TTM. To reduce
the cost per step, we develop a simple, but powerful improvement to a recent
layer distillation method that improves learning via better preserving hidden
state variance. Finally, we combine our step and layer distillation methods
together for a dual-faceted approach. We evaluate our step and layer
distillation methods independently and show each yield best-in-class
performance. Our combined distillation method can generate high-quality outputs
with improved diversity, accelerating our base model by 10-18x (230/435ms
latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) --
the fastest high-quality TTM to our knowledge. Sound examples can be found at
https://presto-music.github.io/web/.
Presto! accelerates score-based diffusion transformers in text-to-music generation through step and layer distillation, achieving high-quality and fast outputs.
diffusion-based text-to-music diffusion transformers score-based distribution matching distillation EDM-family GAN-based distillation hidden state variance dual-faceted approach