  The audio-video based multimodal emotion recognition has attracted a lot of
attention due to its robust performance. Most of the existing methods focus on
proposing different cross-modal fusion strategies. However, these strategies
introduce redundancy in the features of different modalities without fully
considering the complementary properties between modal information, and these
approaches do not guarantee the non-loss of original semantic information
during intra- and inter-modal interactions. In this paper, we propose a novel
cross-modal fusion network based on self-attention and residual structure
(CFN-SR) for multimodal emotion recognition. Firstly, we perform representation
learning for audio and video modalities to obtain the semantic features of the
two modalities by efficient ResNeXt and 1D CNN, respectively. Secondly, we feed
the features of the two modalities into the cross-modal blocks separately to
ensure efficient complementarity and completeness of information through the
self-attention mechanism and residual structure. Finally, we obtain the output
of emotions by splicing the obtained fused representation with the original
representation. To verify the effectiveness of the proposed method, we conduct
experiments on the RAVDESS dataset. The experimental results show that the
proposed CFN-SR achieves the state-of-the-art and obtains 75.76% accuracy with
26.30M parameters. Our code is available at
https://github.com/skeletonNN/CFN-SR.
