  Transformers are groundbreaking architectures that have changed a flow of
deep learning, and many high-performance models are developing based on
transformer architectures. Transformers implemented only with attention with
encoder-decoder structure following seq2seq without using RNN, but had better
performance than RNN. Herein, we investigate the decoding technique for
electroencephalography (EEG) composed of self-attention module from transformer
architecture during imagined speech and overt speech. We performed
classification of nine subjects using convolutional neural network based on
EEGNet that captures temporal-spectral-spatial features from EEG of imagined
speech and overt speech. Furthermore, we applied the self-attention module to
decoding EEG to improve the performance and lower the number of parameters. Our
results demonstrate the possibility of decoding brain activities of imagined
speech and overt speech using attention modules. Also, only single channel EEG
or ear-EEG can be used to decode the imagined speech for practical BCIs.
