Seeing through the Brain: Image Reconstruction of Visual Perception from
  Human Brain Signals
Seeing is believing, however, the underlying mechanism of how human visual
perceptions are intertwined with our cognitions is still a mystery. Thanks to
the recent advances in both neuroscience and artificial intelligence, we have
been able to record the visually evoked brain activities and mimic the visual
perception ability through computational approaches. In this paper, we pay
attention to visual stimuli reconstruction by reconstructing the observed
images based on portably accessible brain signals, i.e., electroencephalography
(EEG) data. Since EEG signals are dynamic in the time-series format and are
notorious to be noisy, processing and extracting useful information requires
more dedicated efforts; In this paper, we propose a comprehensive pipeline,
named NeuroImagen, for reconstructing visual stimuli images from EEG signals.
Specifically, we incorporate a novel multi-level perceptual information
decoding to draw multi-grained outputs from the given EEG data. A latent
diffusion model will then leverage the extracted information to reconstruct the
high-resolution visual stimuli images. The experimental results have
illustrated the effectiveness of image reconstruction and superior quantitative
performance of our proposed method.
64d1d53b44d373d7067453ac
Yu-Ting Lan
64d1d53b44d373d7067453ad
Kan Ren
64d1d53b44d373d7067453ae
Yansen Wang
64d1d53b44d373d7067453af
Wei-Long Zheng
64d1d53b44d373d7067453b0
Dongsheng Li
64d1d53b44d373d7067453b1
Bao-Liang Lu
64d1d53b44d373d7067453b2
Lili Qiu
Seeing through the Brain: Image Reconstruction of Visual Perception from
  Human Brain Signals
Seeing is believing, however, the underlying mechanism of how human visual
perceptions are intertwined with our cognitions is still a mystery. Thanks to
the recent advances in both neuroscience and artificial intelligence, we have
been able to record the visually evoked brain activities and mimic the visual
perception ability through computational approaches. In this paper, we pay
attention to visual stimuli reconstruction by reconstructing the observed
images based on portably accessible brain signals, i.e., electroencephalography
(EEG) data. Since EEG signals are dynamic in the time-series format and are
notorious to be noisy, processing and extracting useful information requires
more dedicated efforts; In this paper, we propose a comprehensive pipeline,
named NeuroImagen, for reconstructing visual stimuli images from EEG signals.
Specifically, we incorporate a novel multi-level perceptual information
decoding to draw multi-grained outputs from the given EEG data. A latent
diffusion model will then leverage the extracted information to reconstruct the
high-resolution visual stimuli images. The experimental results have
illustrated the effectiveness of image reconstruction and superior quantitative
performance of our proposed method.
A pipeline named NeuroImagen uses EEG signals and a latent diffusion model to reconstruct high-resolution visual images, demonstrating superior performance in image reconstruction.
visual stimuli reconstruction electroencephalography EEG multi-level perceptual information decoding latent diffusion model high-resolution visual stimuli images