Naturalistic Music Decoding from EEG Data via Latent Diffusion Models
In this article, we explore the potential of using latent diffusion models, a
family of powerful generative models, for the task of reconstructing
naturalistic music from electroencephalogram (EEG) recordings. Unlike simpler
music with limited timbres, such as MIDI-generated tunes or monophonic pieces,
the focus here is on intricate music featuring a diverse array of instruments,
voices, and effects, rich in harmonics and timbre. This study represents an
initial foray into achieving general music reconstruction of high-quality using
non-invasive EEG data, employing an end-to-end training approach directly on
raw data without the need for manual pre-processing and channel selection. We
train our models on the public NMED-T dataset and perform quantitative
evaluation proposing neural embedding-based metrics. We additionally perform
song classification based on the generated tracks. Our work contributes to the
ongoing research in neural decoding and brain-computer interfaces, offering
insights into the feasibility of using EEG data for complex auditory
information reconstruction.
66460e85dfab02c1ba3f439d
Emilian Postolache
claimed_verified
2024-05-20T07:38:47.672Z
630a39aafaaea618ebc28bf3
/avatars/6b3e116a2901c842dfc19935fdf7ba7a.svg
Postolache
user
Emilian
66460e85dfab02c1ba3f439e
Natalia Polouliakh
66460e85dfab02c1ba3f439f
Hiroaki Kitano
66460e85dfab02c1ba3f43a0
Akima Connelly
66460e85dfab02c1ba3f43a1
Emanuele Rodolà
66460e85dfab02c1ba3f43a2
Taketo Akama
Naturalistic Music Decoding from EEG Data via Latent Diffusion Models
In this article, we explore the potential of using latent diffusion models, a
family of powerful generative models, for the task of reconstructing
naturalistic music from electroencephalogram (EEG) recordings. Unlike simpler
music with limited timbres, such as MIDI-generated tunes or monophonic pieces,
the focus here is on intricate music featuring a diverse array of instruments,
voices, and effects, rich in harmonics and timbre. This study represents an
initial foray into achieving general music reconstruction of high-quality using
non-invasive EEG data, employing an end-to-end training approach directly on
raw data without the need for manual pre-processing and channel selection. We
train our models on the public NMED-T dataset and perform quantitative
evaluation proposing neural embedding-based metrics. We additionally perform
song classification based on the generated tracks. Our work contributes to the
ongoing research in neural decoding and brain-computer interfaces, offering
insights into the feasibility of using EEG data for complex auditory
information reconstruction.
Latent diffusion models are employed to reconstruct naturalistic music from EEG recordings using an end-to-end training approach and evaluating with neural embedding-based metrics.
latent diffusion models generative models music reconstruction EEG recordings NMED-T dataset neural embedding-based metrics song classification neural decoding brain-computer interfaces