LP-MusicCaps: LLM-Based Pseudo Music Captioning
Automatic music captioning, which generates natural language descriptions for
given music tracks, holds significant potential for enhancing the understanding
and organization of large volumes of musical data. Despite its importance,
researchers face challenges due to the costly and time-consuming collection
process of existing music-language datasets, which are limited in size. To
address this data scarcity issue, we propose the use of large language models
(LLMs) to artificially generate the description sentences from large-scale tag
datasets. This results in approximately 2.2M captions paired with 0.5M audio
clips. We term it Large Language Model based Pseudo music caption dataset,
shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale
music captioning dataset with various quantitative evaluation metrics used in
the field of natural language processing as well as human evaluation. In
addition, we trained a transformer-based music captioning model with the
dataset and evaluated it under zero-shot and transfer-learning settings. The
results demonstrate that our proposed approach outperforms the supervised
baseline model.
64c8694e3cfe45b07147b1ad
SeungHeon Doh
claimed_verified
2023-08-01T07:44:29.602Z
637c3504c292c0fd3f37361f
https://cdn-avatars.huggingface.co/v1/production/uploads/637c3504c292c0fd3f37361f/wyTkbYKi8HufRT65LGN0P.jpeg
seungheon.doh
user
seungheondoh
64c8694e3cfe45b07147b1ae
Keunwoo Choi
admin_assigned
2023-08-01T09:24:50.173Z
63c4c4bbc7d7f4c63a624db1
https://cdn-avatars.huggingface.co/v1/production/uploads/63c4c4bbc7d7f4c63a624db1/Uvy-EOA_LllinXfjHJx-e.jpeg
Keunwoo choi
user
keunwoochoi
64c8694e3cfe45b07147b1af
Jongpil Lee
64c8694e3cfe45b07147b1b0
Juhan Nam
claimed_verified
2023-08-07T12:38:57.409Z
64cf3e0c9c245c6ba728c831
/avatars/740b7c3a25568b5556cc34104322209f.svg
Juhan Nam
user
juhannam
LP-MusicCaps: LLM-Based Pseudo Music Captioning
Automatic music captioning, which generates natural language descriptions for
given music tracks, holds significant potential for enhancing the understanding
and organization of large volumes of musical data. Despite its importance,
researchers face challenges due to the costly and time-consuming collection
process of existing music-language datasets, which are limited in size. To
address this data scarcity issue, we propose the use of large language models
(LLMs) to artificially generate the description sentences from large-scale tag
datasets. This results in approximately 2.2M captions paired with 0.5M audio
clips. We term it Large Language Model based Pseudo music caption dataset,
shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale
music captioning dataset with various quantitative evaluation metrics used in
the field of natural language processing as well as human evaluation. In
addition, we trained a transformer-based music captioning model with the
dataset and evaluated it under zero-shot and transfer-learning settings. The
results demonstrate that our proposed approach outperforms the supervised
baseline model.
A large-scale pseudo music caption dataset generated using LLMs improves music captioning performance over supervised models in zero-shot and transfer-learning settings.
large language models LLMs LP-MusicCaps transformer-based music captioning model zero-shot transfer-learning