Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesis
In text-to-speech (TTS) synthesis, diffusion models have achieved promising
generation quality. However, because of the pre-defined data-to-noise diffusion
process, their prior distribution is restricted to a noisy representation,
which provides little information of the generation target. In this work, we
present a novel TTS system, Bridge-TTS, making the first attempt to substitute
the noisy Gaussian prior in established diffusion-based TTS methods with a
clean and deterministic one, which provides strong structural information of
the target. Specifically, we leverage the latent representation obtained from
text input as our prior, and build a fully tractable Schrodinger bridge between
it and the ground-truth mel-spectrogram, leading to a data-to-data process.
Moreover, the tractability and flexibility of our formulation allow us to
empirically study the design spaces such as noise schedules, as well as to
develop stochastic and deterministic samplers. Experimental results on the
LJ-Speech dataset illustrate the effectiveness of our method in terms of both
synthesis quality and sampling efficiency, significantly outperforming our
diffusion counterpart Grad-TTS in 50-step/1000-step synthesis and strong fast
TTS models in few-step scenarios. Project page: https://bridge-tts.github.io/
65713f7ad877298e1d5379a1
Zehua Chen
admin_assigned
2023-12-07T10:55:21.695Z
63e411618083f19a218bef49
/avatars/d63fb9ce5e97e2c77e500b329aeec8b0.svg
Zehua Chen
user
ZehuaChenICL
65713f7ad877298e1d5379a2
Guande He
claimed_verified
2024-11-29T10:08:52.053Z
67492ee82ad3cfc108a41bbb
/avatars/7ad03e55a8791c62f1271a5c9bf8cc60.svg
Guande He
user
gdhe17
65713f7ad877298e1d5379a3
Kaiwen Zheng
claimed_verified
2023-12-07T08:29:45.889Z
652bf7edc3cba555d5673c6e
/avatars/78f6416c30203b30671f8423f061c657.svg
Kaiwen Zheng
user
worstcoder
65713f7ad877298e1d5379a4
Xu Tan
admin_assigned
2023-12-07T10:55:39.819Z
5f1040b6e9d71719e3be71d2
/avatars/a2f28940236ae625ed3810ad62e343ff.svg
Xu Tan
user
xutan
65713f7ad877298e1d5379a5
Jun Zhu
Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesis
In text-to-speech (TTS) synthesis, diffusion models have achieved promising
generation quality. However, because of the pre-defined data-to-noise diffusion
process, their prior distribution is restricted to a noisy representation,
which provides little information of the generation target. In this work, we
present a novel TTS system, Bridge-TTS, making the first attempt to substitute
the noisy Gaussian prior in established diffusion-based TTS methods with a
clean and deterministic one, which provides strong structural information of
the target. Specifically, we leverage the latent representation obtained from
text input as our prior, and build a fully tractable Schrodinger bridge between
it and the ground-truth mel-spectrogram, leading to a data-to-data process.
Moreover, the tractability and flexibility of our formulation allow us to
empirically study the design spaces such as noise schedules, as well as to
develop stochastic and deterministic samplers. Experimental results on the
LJ-Speech dataset illustrate the effectiveness of our method in terms of both
synthesis quality and sampling efficiency, significantly outperforming our
diffusion counterpart Grad-TTS in 50-step/1000-step synthesis and strong fast
TTS models in few-step scenarios. Project page: https://bridge-tts.github.io/
Bridge-TTS introduces a novel text-to-speech system using a clean deterministic prior in diffusion models, significantly improving synthesis quality and sampling efficiency over existing methods.
diffusion models noisy Gaussian prior clean and deterministic prior latent representation Schrodinger bridge ground-truth mel-spectrogram data-to-data process noise schedules stochastic samplers deterministic samplers LJ-Speech dataset Grad-TTS