  The lack of data and the difficulty of multimodal fusion have always been
challenges for multimodal emotion recognition (MER). In this paper, we propose
to use pretrained models as upstream network, wav2vec 2.0 for audio modality
and BERT for text modality, and finetune them in downstream task of MER to cope
with the lack of data. For the difficulty of multimodal fusion, we use a
K-layer multi-head attention mechanism as a downstream fusion module. Starting
from the MER task itself, we design two auxiliary tasks to alleviate the
insufficient fusion between modalities and guide the network to capture and
align emotion-related features. Compared to the previous state-of-the-art
models, we achieve a better performance by 78.42% Weighted Accuracy (WA) and
79.71% Unweighted Accuracy (UA) on the IEMOCAP dataset.
