  Recent advances in deep learning have had a methodological and practical
impact on brain-computer interface research. Among the various deep network
architectures, convolutional neural networks have been well suited for
spatio-spectral-temporal electroencephalogram signal representation learning.
Most of the existing CNN-based methods described in the literature extract
features at a sequential level of abstraction with repetitive nonlinear
operations and involve densely connected layers for classification. However,
studies in neurophysiology have revealed that EEG signals carry information in
different ranges of frequency components. To better reflect these
multi-frequency properties in EEGs, we propose a novel deep multi-scale neural
network that discovers feature representations in multiple frequency/time
ranges and extracts relationships among electrodes, i.e., spatial
representations, for subject intention/condition identification. Furthermore,
by completely representing EEG signals with spatio-spectral-temporal
information, the proposed method can be utilized for diverse paradigms in both
active and passive BCIs, contrary to existing methods that are primarily
focused on single-paradigm BCIs. To demonstrate the validity of our proposed
method, we conducted experiments on various paradigms of active/passive BCI
datasets. Our experimental results demonstrated that the proposed method
achieved performance improvements when judged against comparable
state-of-the-art methods. Additionally, we analyzed the proposed method using
different techniques, such as PSD curves and relevance score inspection to
validate the multi-scale EEG signal information capturing ability, activation
pattern maps for investigating the learned spatial filters, and t-SNE plotting
for visualizing represented features. Finally, we also demonstrated our
method's application to real-world problems.
