Better speech synthesis through scaling
In recent years, the field of image generation has been revolutionized by the
application of autoregressive transformers and DDPMs. These approaches model
the process of image generation as a step-wise probabilistic processes and
leverage large amounts of compute and data to learn the image distribution.
This methodology of improving performance need not be confined to images. This
paper describes a way to apply advances in the image generative domain to
speech synthesis. The result is TorToise -- an expressive, multi-voice
text-to-speech system.
  All model code and trained weights have been open-sourced at
https://github.com/neonbjb/tortoise-tts.
64617fc2cf638aa8f855e05c
James Betker
claimed_verified
2023-05-16T15:32:44.928Z
61eafd948ae3ccab3b3da96d
https://cdn-avatars.huggingface.co/v1/production/uploads/1650321610784-61eafd948ae3ccab3b3da96d.jpeg
James Betker
user
jbetker
Better speech synthesis through scaling
In recent years, the field of image generation has been revolutionized by the
application of autoregressive transformers and DDPMs. These approaches model
the process of image generation as a step-wise probabilistic processes and
leverage large amounts of compute and data to learn the image distribution.
This methodology of improving performance need not be confined to images. This
paper describes a way to apply advances in the image generative domain to
speech synthesis. The result is TorToise -- an expressive, multi-voice
text-to-speech system.
  All model code and trained weights have been open-sourced at
https://github.com/neonbjb/tortoise-tts.
TorToise is an expressive multi-voice speech synthesis system that applies autoregressive transformers and DDPMs from image generation to text-to-speech.
autoregressive transformers DDPMs