  Decoding information from bio-signals such as EEG, using machine learning has
been a challenge due to the small data-sets and difficulty to obtain labels. We
propose a reconstruction-based self-supervised learning model, the masked
auto-encoder for EEG (MAEEG), for learning EEG representations by learning to
reconstruct the masked EEG features using a transformer architecture. We found
that MAEEG can learn representations that significantly improve sleep stage
classification (~5% accuracy increase) when only a small number of labels are
given. We also found that input sample lengths and different ways of masking
during reconstruction-based SSL pretraining have a huge effect on downstream
model performance. Specifically, learning to reconstruct a larger proportion
and more concentrated masked signal results in better performance on sleep
classification. Our findings provide insight into how reconstruction-based SSL
could help representation learning for EEG.
