  In this paper, we propose a method to improve the accuracy of speech emotion
recognition (SER) by using vision transformer (ViT) to attend to the
correlation of frequency (y-axis) with time (x-axis) in spectrogram and
transferring positional information between ViT through knowledge transfer. The
proposed method has the following originality i) We use vertically segmented
patches of log-Mel spectrogram to analyze the correlation of frequencies over
time. This type of patch allows us to correlate the most relevant frequencies
for a particular emotion with the time they were uttered. ii) We propose the
use of image coordinate encoding, an absolute positional encoding suitable for
ViT. By normalizing the x, y coordinates of the image to -1 to 1 and
concatenating them to the image, we can effectively provide valid absolute
positional information for ViT. iii) Through feature map matching, the locality
and location information of the teacher network is effectively transmitted to
the student network. Teacher network is a ViT that contains locality of
convolutional stem and absolute position information through image coordinate
encoding, and student network is a structure that lacks positional encoding in
the basic ViT structure. In feature map matching stage, we train through the
mean absolute error (L1 loss) to minimize the difference between the feature
maps of the two networks. To validate the proposed method, three emotion
datasets (SAVEE, EmoDB, and CREMA-D) consisting of speech were converted into
log-Mel spectrograms for comparison experiments. The experimental results show
that the proposed method significantly outperforms the state-of-the-art methods
in terms of weighted accuracy while requiring significantly fewer floating
point operations (FLOPs). Overall, the proposed method offers an promising
solution for SER by providing improved efficiency and performance.
