MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for
  Zero-Shot Speech Synthesis
While recent zero-shot text-to-speech (TTS) models have significantly
improved speech quality and expressiveness, mainstream systems still suffer
from issues related to speech-text alignment modeling: 1) models without
explicit speech-text alignment modeling exhibit less robustness, especially for
hard sentences in practical applications; 2) predefined alignment-based models
suffer from naturalness constraints of forced alignments. This paper introduces
MegaTTS 3, a TTS system featuring an innovative sparse alignment
algorithm that guides the latent diffusion transformer (DiT). Specifically, we
provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of
alignment without limiting the search space, thereby achieving high
naturalness. Moreover, we employ a multi-condition classifier-free guidance
strategy for accent intensity adjustment and adopt the piecewise rectified flow
technique to accelerate the generation process. Experiments demonstrate that
MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports
highly flexible control over accent intensity. Notably, our system can generate
high-quality one-minute speech with only 8 sampling steps. Audio samples are
available at https://sditdemo.github.io/sditdemo/.
67ee3cf7042e8ba3e96fc1e0
Ziyue Jiang
claimed_verified
2025-04-03T08:28:36.564Z
67dbdf261956dcedf0f0a7e1
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/BRzXe_7jytEYadJ0byMyD.png
ZiyueJiang
user
ZiyueJiang
67ee3cf7042e8ba3e96fc1e1
Yi Ren
67ee3cf7042e8ba3e96fc1e2
Ruiqi Li
67ee3cf7042e8ba3e96fc1e3
Shengpeng Ji
67ee3cf7042e8ba3e96fc1e4
Boyang Zhang
67ee3cf7042e8ba3e96fc1e5
Zhenhui Ye
67ee3cf7042e8ba3e96fc1e6
Chen Zhang
67ee3cf7042e8ba3e96fc1e7
Bai Jionghao
67ee3cf7042e8ba3e96fc1e8
Xiaoda Yang
67ee3cf7042e8ba3e96fc1e9
Jialong Zuo
67ee3cf7042e8ba3e96fc1ea
Yu Zhang
claimed_verified
2025-04-30T09:56:50.803Z
66569729ea21cfae5f5797c4
https://cdn-avatars.huggingface.co/v1/production/uploads/66569729ea21cfae5f5797c4/IguwJzljFN3QiEd1bn5BP.jpeg
Yu Zhang
user
AaronZ345
67ee3cf7042e8ba3e96fc1eb
Rui Liu
67ee3cf7042e8ba3e96fc1ec
Xiang Yin
67ee3cf7042e8ba3e96fc1ed
Zhou Zhao
MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for
  Zero-Shot Speech Synthesis
While recent zero-shot text-to-speech (TTS) models have significantly
improved speech quality and expressiveness, mainstream systems still suffer
from issues related to speech-text alignment modeling: 1) models without
explicit speech-text alignment modeling exhibit less robustness, especially for
hard sentences in practical applications; 2) predefined alignment-based models
suffer from naturalness constraints of forced alignments. This paper introduces
MegaTTS 3, a TTS system featuring an innovative sparse alignment
algorithm that guides the latent diffusion transformer (DiT). Specifically, we
provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of
alignment without limiting the search space, thereby achieving high
naturalness. Moreover, we employ a multi-condition classifier-free guidance
strategy for accent intensity adjustment and adopt the piecewise rectified flow
technique to accelerate the generation process. Experiments demonstrate that
MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports
highly flexible control over accent intensity. Notably, our system can generate
high-quality one-minute speech with only 8 sampling steps. Audio samples are
available at https://sditdemo.github.io/sditdemo/.
MegaTTS 3, a zero-shot text-to-speech system, uses a sparse alignment algorithm and latent diffusion transformer for high-quality speech generation with flexible accent control.
sparse alignment algorithm latent diffusion transformer multi-condition classifier-free guidance piecewise rectified flow