SonicSim: A customizable simulation platform for speech processing in
  moving sound source scenarios
The systematic evaluation of speech separation and enhancement models under
moving sound source conditions typically requires extensive data comprising
diverse scenarios. However, real-world datasets often contain insufficient data
to meet the training and evaluation requirements of models. Although synthetic
datasets offer a larger volume of data, their acoustic simulations lack
realism. Consequently, neither real-world nor synthetic datasets effectively
fulfill practical needs. To address these issues, we introduce SonicSim, a
synthetic toolkit de-designed to generate highly customizable data for moving
sound sources. SonicSim is developed based on the embodied AI simulation
platform, Habitat-sim, supporting multi-level adjustments, including
scene-level, microphone-level, and source-level, thereby generating more
diverse synthetic data. Leveraging SonicSim, we constructed a moving sound
source benchmark dataset, SonicSet, using the Librispeech, the Freesound
Dataset 50k (FSD50K) and Free Music Archive (FMA), and 90 scenes from the
Matterport3D to evaluate speech separation and enhancement models.
Additionally, to validate the differences between synthetic data and real-world
data, we randomly selected 5 hours of raw data without reverberation from the
SonicSet validation set to record a real-world speech separation dataset, which
was then compared with the corresponding synthetic datasets. Similarly, we
utilized the real-world speech enhancement dataset RealMAN to validate the
acoustic gap between other synthetic datasets and the SonicSet dataset for
speech enhancement. The results indicate that the synthetic data generated by
SonicSim can effectively generalize to real-world scenarios. Demo and code are
publicly available at https://cslikai.cn/SonicSim/.
66feb6df001816b29e69cd78
Kai Li
extracted_pending
2024-10-03T15:23:13.186Z
6387676c23da90491eb9fb16
https://cdn-avatars.huggingface.co/v1/production/uploads/1669818175965-noauth.jpeg
Kai Li
user
JusperLee
66feb6df001816b29e69cd79
Wendi Sang
66feb6df001816b29e69cd7a
Chang Zeng
66feb6df001816b29e69cd7b
Runxuan Yang
66feb6df001816b29e69cd7c
Guo Chen
66feb6df001816b29e69cd7d
Xiaolin Hu
SonicSim: A customizable simulation platform for speech processing in
  moving sound source scenarios
The systematic evaluation of speech separation and enhancement models under
moving sound source conditions typically requires extensive data comprising
diverse scenarios. However, real-world datasets often contain insufficient data
to meet the training and evaluation requirements of models. Although synthetic
datasets offer a larger volume of data, their acoustic simulations lack
realism. Consequently, neither real-world nor synthetic datasets effectively
fulfill practical needs. To address these issues, we introduce SonicSim, a
synthetic toolkit de-designed to generate highly customizable data for moving
sound sources. SonicSim is developed based on the embodied AI simulation
platform, Habitat-sim, supporting multi-level adjustments, including
scene-level, microphone-level, and source-level, thereby generating more
diverse synthetic data. Leveraging SonicSim, we constructed a moving sound
source benchmark dataset, SonicSet, using the Librispeech, the Freesound
Dataset 50k (FSD50K) and Free Music Archive (FMA), and 90 scenes from the
Matterport3D to evaluate speech separation and enhancement models.
Additionally, to validate the differences between synthetic data and real-world
data, we randomly selected 5 hours of raw data without reverberation from the
SonicSet validation set to record a real-world speech separation dataset, which
was then compared with the corresponding synthetic datasets. Similarly, we
utilized the real-world speech enhancement dataset RealMAN to validate the
acoustic gap between other synthetic datasets and the SonicSet dataset for
speech enhancement. The results indicate that the synthetic data generated by
SonicSim can effectively generalize to real-world scenarios. Demo and code are
publicly available at https://cslikai.cn/SonicSim/.
SonicSim is a synthetic toolkit generating customizable data for moving sound sources, enhancing speech separation and enhancement models' evaluation.
speech separation speech enhancement synthetic datasets Habitat-sim scene-level microphone-level source-level moving sound source benchmark SonicSet Librispeech FSD50K FMA Matterport3D