BASE TTS: Lessons from building a billion-parameter Text-to-Speech model
  on 100K hours of data
We introduce a text-to-speech (TTS) model called BASE TTS, which stands for
Big Adaptive Streamable TTS with
Emergent abilities. BASE TTS is the largest TTS model to-date,
trained on 100K hours of public domain speech data, achieving a new
state-of-the-art in speech naturalness. It deploys a 1-billion-parameter
autoregressive Transformer that converts raw texts into discrete codes
("speechcodes") followed by a convolution-based decoder which converts these
speechcodes into waveforms in an incremental, streamable manner. Further, our
speechcodes are built using a novel speech tokenization technique that features
speaker ID disentanglement and compression with byte-pair encoding. Echoing the
widely-reported "emergent abilities" of large language models when trained on
increasing volume of data, we show that BASE TTS variants built with 10K+ hours
and 500M+ parameters begin to demonstrate natural prosody on textually complex
sentences. We design and share a specialized dataset to measure these emergent
abilities for text-to-speech. We showcase state-of-the-art naturalness of BASE
TTS by evaluating against baselines that include publicly available large-scale
text-to-speech systems: YourTTS, Bark and TortoiseTTS. Audio samples generated
by the model can be heard at https://amazon-ltts-paper.com/.
65cc2d6a11a80579a430d70d
Mateusz Łajszczak
65cc2d6a11a80579a430d70e
Guillermo Cámbara
claimed_verified
2024-02-14T10:11:25.332Z
632862ca27174b0272254f82
https://cdn-avatars.huggingface.co/v1/production/uploads/632862ca27174b0272254f82/TiM-OcoH0jsCQqf0OkWtW.jpeg
Guillermo Cámbara
user
gcambara
65cc2d6a11a80579a430d70f
Yang Li
claimed_verified
2024-02-15T08:28:02.737Z
65cccc8ed4f3ed30d1870e37
https://cdn-avatars.huggingface.co/v1/production/uploads/65cccc8ed4f3ed30d1870e37/0ETlxoPP187u6XFySa4GZ.jpeg
Yang Li
user
lizyacbg
65cc2d6a11a80579a430d710
Fatih Beyhan
admin_assigned
2024-02-14T10:21:13.167Z
5ec973f3968f6028e0559f79
https://cdn-avatars.huggingface.co/v1/production/uploads/1628268177397-5ec973f3968f6028e0559f79.jpeg
Fatih Beyhan
user
fatihbeyhan
65cc2d6a11a80579a430d711
Arent van Korlaar
claimed_verified
2024-02-15T08:28:00.311Z
64ad769fad6218d51a1305a5
/avatars/6fd6cede811190a3e725be2c1ed06eaf.svg
Arent van Korlaar
user
afvk
65cc2d6a11a80579a430d712
Fan Yang
65cc2d6a11a80579a430d713
Arnaud Joly
65cc2d6a11a80579a430d714
Álvaro Martín-Cortinas
65cc2d6a11a80579a430d715
Ammar Abbas
65cc2d6a11a80579a430d716
Adam Michalski
admin_assigned
2024-02-15T08:27:45.676Z
65cd14193f97d5bd5d37f686
/avatars/cbfe60b0f294c6c6632e5f3d349842f0.svg
Adam Michalski
user
micadam
65cc2d6a11a80579a430d717
Alexis Moinet
65cc2d6a11a80579a430d718
Sri Karlapati
65cc2d6a11a80579a430d719
Ewa Muszyńska
65cc2d6a11a80579a430d71a
Haohan Guo
admin_assigned
2024-02-14T10:25:35.635Z
63783d9d84318944acd305c4
/avatars/cafd4fd0287e77cc6ebf37c8c8509174.svg
Haohan Guo
user
hhguo
65cc2d6a11a80579a430d71b
Bartosz Putrycz
65cc2d6a11a80579a430d71c
Soledad López Gambino
65cc2d6a11a80579a430d71d
Kayeon Yoo
65cc2d6a11a80579a430d71e
Elena Sokolova
65cc2d6a11a80579a430d71f
Thomas Drugman
BASE TTS: Lessons from building a billion-parameter Text-to-Speech model
  on 100K hours of data
We introduce a text-to-speech (TTS) model called BASE TTS, which stands for
Big Adaptive Streamable TTS with
Emergent abilities. BASE TTS is the largest TTS model to-date,
trained on 100K hours of public domain speech data, achieving a new
state-of-the-art in speech naturalness. It deploys a 1-billion-parameter
autoregressive Transformer that converts raw texts into discrete codes
("speechcodes") followed by a convolution-based decoder which converts these
speechcodes into waveforms in an incremental, streamable manner. Further, our
speechcodes are built using a novel speech tokenization technique that features
speaker ID disentanglement and compression with byte-pair encoding. Echoing the
widely-reported "emergent abilities" of large language models when trained on
increasing volume of data, we show that BASE TTS variants built with 10K+ hours
and 500M+ parameters begin to demonstrate natural prosody on textually complex
sentences. We design and share a specialized dataset to measure these emergent
abilities for text-to-speech. We showcase state-of-the-art naturalness of BASE
TTS by evaluating against baselines that include publicly available large-scale
text-to-speech systems: YourTTS, Bark and TortoiseTTS. Audio samples generated
by the model can be heard at https://amazon-ltts-paper.com/.
BASE TTS, a large-scale text-to-speech model, demonstrates advanced naturalness and emergent abilities using autoregressive Transformers and novel speech tokenization techniques.
autoregressive Transformer speechcodes convolution-based decoder speech tokenization speaker ID disentanglement byte-pair encoding natural prosody