  Self-supervised learning has produced impressive results in multimedia
domains of audio, vision and speech. This paradigm is equally, if not more,
relevant for the domain of biosignals, owing to the scarcity of labelled data
in such scenarios. The ability to leverage large-scale unlabelled data to learn
robust representations could help improve the performance of numerous inference
tasks on biosignals. Given the inherent domain differences between multimedia
modalities and biosignals, the established objectives for self-supervised
learning may not translate well to this domain. Hence, there is an unmet need
to adapt these methods to biosignal analysis. In this work we propose a
self-supervised model for EEG, which provides robust performance and remarkable
parameter efficiency by using state space-based deep learning architecture. We
also propose a novel knowledge-guided pre-training objective that accounts for
the idiosyncrasies of the EEG signal. The results indicate improved embedding
representation learning and downstream performance compared to prior works on
exemplary tasks. Also, the proposed objective significantly reduces the amount
of pre-training data required to obtain performance equivalent to prior works.
