  The computing power of mobile devices limits the end-user applications in
terms of storage size, processing, memory and energy consumption. These
limitations motivate researchers for the design of more efficient deep models.
On the other hand, self-attention networks based on Transformer architecture
have attracted remarkable interests due to their high parallelization
capabilities and strong performance on a variety of Natural Language Processing
(NLP) applications. Inspired by the Transformer, we propose a tandem
Self-Attention Encoding and Pooling (SAEP) mechanism to obtain a discriminative
speaker embedding given non-fixed length speech utterances. SAEP is a stack of
identical blocks solely relied on self-attention and position-wise feed-forward
networks to create vector representation of speakers. This approach encodes
short-term speaker spectral features into speaker embeddings to be used in
text-independent speaker verification. We have evaluated this approach on both
VoxCeleb1 & 2 datasets. The proposed architecture is able to outperform the
baseline x-vector, and shows competitive performance to some other benchmarks
based on convolutions, with a significant reduction in model size. It employs
94%, 95%, and 73% less parameters compared to ResNet-34, ResNet-50, and
x-vector, respectively. This indicates that the proposed fully attention based
architecture is more efficient in extracting time-invariant features from
speaker utterances.
