  How to decode human vision through neural signals has attracted a
long-standing interest in neuroscience and machine learning. Modern contrastive
learning and generative models improved the performance of fMRI-based visual
decoding and reconstruction. However, the high cost and low temporal resolution
of fMRI limit their applications in brain-computer interfaces (BCIs), prompting
a high need for EEG-based visual reconstruction. In this study, we present an
EEG-based visual reconstruction framework. It consists of a plug-and-play EEG
encoder called the Adaptive Thinking Mapper (ATM), which is aligned with image
embeddings, and a two-stage EEG guidance image generator that first transforms
EEG features into image priors and then reconstructs the visual stimuli with a
pre-trained image generator. Our approach allows EEG embeddings to achieve
superior performance in image classification and retrieval tasks. Our two-stage
image generation strategy vividly reconstructs images seen by humans.
Furthermore, we analyzed the impact of signals from different time windows and
brain regions on decoding and reconstruction. The versatility of our framework
is demonstrated in the magnetoencephalogram (MEG) data modality. We report that
EEG-based visual decoding achieves SOTA performance, highlighting the
portability, low cost, and high temporal resolution of EEG, enabling a wide
range of BCI applications. The code of ATM is available at
https://github.com/dongyangli-del/EEG_Image_decode.
