Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for
  Speech Generation
Recent advancements in speech generation have been driven by the large-scale
training datasets. However, current models fall short of capturing the
spontaneity and variability inherent in real-world human speech, due to their
reliance on audiobook datasets limited to formal read-aloud speech styles. To
bridge this gap, we introduce Emilia-Pipe, an open-source preprocessing
pipeline to extract high-quality training data from valuable yet underexplored
in-the-wild data that capture spontaneous human speech in real-world contexts.
By leveraging Emilia-Pipe, we construct Emilia, the first multilingual speech
generation dataset derived from in-the-wild speech data. This dataset comprises
over 101k hours of speech across six languages: English, Chinese, German,
French, Japanese, and Korean. Besides, we expand Emilia to Emilia-Large, a
dataset exceeding 216k hours, making it the largest open-source speech
generation dataset available. Extensive experiments demonstrate that Emilia
significantly outperforms traditional audiobook datasets in generating
spontaneous and human-like speech, showcasing superior performance in capturing
diverse speaker timbre and speaking styles of real-world human speech.
Furthermore, this work underscores the importance of scaling dataset size to
advance speech generation research and validates the effectiveness of Emilia
for both multilingual and crosslingual speech generation.
6798a917a8b0d165e39e17f5
Haorui He
claimed_verified
2025-01-28T13:52:52.095Z
61a7569eaf0333e76eb428a8
https://cdn-avatars.huggingface.co/v1/production/uploads/61a7569eaf0333e76eb428a8/zwseNheR4Hx0DtCmf_v5H.jpeg
HarryHe11
user
HarryHe
6798a917a8b0d165e39e17f6
Zengqiang Shang
admin_assigned
2025-01-28T14:12:12.565Z
64b77dd308e2452d18ddd279
/avatars/258f21fa20a3a187050d80c6088a1f50.svg
shangzengqiang
user
clatter-1
6798a917a8b0d165e39e17f7
Chaoren Wang
6798a917a8b0d165e39e17f8
Xuyuan Li
6798a917a8b0d165e39e17f9
Yicheng Gu
admin_assigned
2025-01-28T14:12:57.932Z
66b5f38a080d890d1727a2a4
/avatars/4d73017ce888437225d994d8ba370e5d.svg
guyicheng
user
guyicheng
6798a917a8b0d165e39e17fa
Hua Hua
6798a917a8b0d165e39e17fb
Liwei Liu
6798a917a8b0d165e39e17fc
Chen Yang
6798a917a8b0d165e39e17fd
Jiaqi Li
admin_assigned
2025-01-28T14:14:30.597Z
6635a711a5243c9638f5e4df
/avatars/08651622fc1fd5089551b510be8c4530.svg
Jiaqi Li
user
jiaqili3
6798a917a8b0d165e39e17fe
Peiyang Shi
6798a917a8b0d165e39e17ff
Yuancheng Wang
admin_assigned
2025-01-28T14:14:50.950Z
63072d60cd148dbc5e49f4dd
/avatars/ffa61038c0ff20848fbcde7c1c34570e.svg
Yuancheng Wang
user
Hecheng0625
6798a917a8b0d165e39e1800
Kai Chen
6798a917a8b0d165e39e1801
Pengyuan Zhang
admin_assigned
2025-01-28T14:14:57.414Z
65fbe8eb030389a29b87446f
/avatars/6d3ba153c41945e566b7c2c2d6af6da6.svg
pengyuan zhang
user
pengyuan2024
6798a917a8b0d165e39e1802
Zhizheng Wu
Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for
  Speech Generation
Recent advancements in speech generation have been driven by the large-scale
training datasets. However, current models fall short of capturing the
spontaneity and variability inherent in real-world human speech, due to their
reliance on audiobook datasets limited to formal read-aloud speech styles. To
bridge this gap, we introduce Emilia-Pipe, an open-source preprocessing
pipeline to extract high-quality training data from valuable yet underexplored
in-the-wild data that capture spontaneous human speech in real-world contexts.
By leveraging Emilia-Pipe, we construct Emilia, the first multilingual speech
generation dataset derived from in-the-wild speech data. This dataset comprises
over 101k hours of speech across six languages: English, Chinese, German,
French, Japanese, and Korean. Besides, we expand Emilia to Emilia-Large, a
dataset exceeding 216k hours, making it the largest open-source speech
generation dataset available. Extensive experiments demonstrate that Emilia
significantly outperforms traditional audiobook datasets in generating
spontaneous and human-like speech, showcasing superior performance in capturing
diverse speaker timbre and speaking styles of real-world human speech.
Furthermore, this work underscores the importance of scaling dataset size to
advance speech generation research and validates the effectiveness of Emilia
for both multilingual and crosslingual speech generation.
Emilia-Pipe preprocessing pipeline creates Emilia and Emilia-Large datasets, enhancing speech generation with high-quality in-the-wild multilingual data, surpassing audiobook datasets.
speech generation Emilia-Pipe multilingual dataset in-the-wild Emilia Emilia-Large speaker timbre speaking styles