  Automatic speech emotion recognition (SER) is a challenging task that plays a
crucial role in natural human-computer interaction. One of the main challenges
in SER is data scarcity, i.e., insufficient amounts of carefully labeled data
to build and fully explore complex deep learning models for emotion
classification. This paper aims to address this challenge using a transfer
learning strategy combined with spectrogram augmentation. Specifically, we
propose a transfer learning approach that leverages a pre-trained residual
network (ResNet) model including a statistics pooling layer from speaker
recognition trained using large amounts of speaker-labeled data. The statistics
pooling layer enables the model to efficiently process variable-length input,
thereby eliminating the need for sequence truncation which is commonly used in
SER systems. In addition, we adopt a spectrogram augmentation technique to
generate additional training data samples by applying random time-frequency
masks to log-mel spectrograms to mitigate overfitting and improve the
generalization of emotion recognition models. We evaluate the effectiveness of
our proposed approach on the interactive emotional dyadic motion capture
(IEMOCAP) dataset. Experimental results indicate that the transfer learning and
spectrogram augmentation approaches improve the SER performance, and when
combined achieve state-of-the-art results.
