A Large-scale Dataset for Audio-Language Representation Learning
The AI community has made significant strides in developing powerful
foundation models, driven by large-scale multimodal datasets. However, in the
audio representation learning community, the present audio-language datasets
suffer from limitations such as insufficient volume, simplistic content, and
arduous collection procedures. To tackle these challenges, we present an
innovative and automatic audio caption generation pipeline based on a series of
public tools or APIs, and construct a large-scale, high-quality, audio-language
dataset, named as Auto-ACD, comprising over 1.9M audio-text pairs. To
demonstrate the effectiveness of the proposed dataset, we train popular models
on our dataset and show performance improvement on various downstream tasks,
namely, audio-language retrieval, audio captioning, environment classification.
In addition, we establish a novel test set and provide a benchmark for
audio-text tasks. The proposed dataset will be released at
https://auto-acd.github.io/.
650bd248b92f35bd37d263c1
Luoyi Sun
650bd248b92f35bd37d263c2
Xuenan Xu
admin_assigned
2023-09-21T09:50:04.592Z
6457623c90bf61c87f9ffdc4
https://cdn-avatars.huggingface.co/v1/production/uploads/6457623c90bf61c87f9ffdc4/pKG76YipWfYFhSnorepsl.jpeg
Xuenan Xu
user
wsntxxn
650bd248b92f35bd37d263c3
Mengyue Wu
650bd248b92f35bd37d263c4
Weidi Xie
admin_assigned
2023-09-21T09:50:24.463Z
63f1841f2f7c0152e8748ec4
/avatars/5398658c7d0cf556531a625a4ca5d18a.svg
Xie
user
Weidi
A Large-scale Dataset for Audio-Language Representation Learning
The AI community has made significant strides in developing powerful
foundation models, driven by large-scale multimodal datasets. However, in the
audio representation learning community, the present audio-language datasets
suffer from limitations such as insufficient volume, simplistic content, and
arduous collection procedures. To tackle these challenges, we present an
innovative and automatic audio caption generation pipeline based on a series of
public tools or APIs, and construct a large-scale, high-quality, audio-language
dataset, named as Auto-ACD, comprising over 1.9M audio-text pairs. To
demonstrate the effectiveness of the proposed dataset, we train popular models
on our dataset and show performance improvement on various downstream tasks,
namely, audio-language retrieval, audio captioning, environment classification.
In addition, we establish a novel test set and provide a benchmark for
audio-text tasks. The proposed dataset will be released at
https://auto-acd.github.io/.
An automatic pipeline creates a large audio-text dataset, Auto-ACD, improving performance on audio-language retrieval, captioning, and classification tasks.
audio representation learning audio-language datasets audio caption generation audio-language retrieval audio captioning environment classification