  Though multimodal emotion recognition has achieved significant progress over
recent years, the potential of rich synergic relationships across the
modalities is not fully exploited. In this paper, we introduce Recursive Joint
Cross-Modal Attention (RJCMA) to effectively capture both intra- and
inter-modal relationships across audio, visual, and text modalities for
dimensional emotion recognition. In particular, we compute the attention
weights based on cross-correlation between the joint audio-visual-text feature
representations and the feature representations of individual modalities to
simultaneously capture intra- and intermodal relationships across the
modalities. The attended features of the individual modalities are again fed as
input to the fusion model in a recursive mechanism to obtain more refined
feature representations. We have also explored Temporal Convolutional Networks
(TCNs) to improve the temporal modeling of the feature representations of
individual modalities. Extensive experiments are conducted to evaluate the
performance of the proposed fusion model on the challenging Affwild2 dataset.
By effectively capturing the synergic intra- and inter-modal relationships
across audio, visual, and text modalities, the proposed fusion model achieves a
Concordance Correlation Coefficient (CCC) of 0.585 (0.542) and 0.674 (0.619)
for valence and arousal respectively on the validation set(test set). This
shows a significant improvement over the baseline of 0.240 (0.211) and 0.200
(0.191) for valence and arousal, respectively, in the validation set (test
set), achieving second place in the valence-arousal challenge of the 6th
Affective Behavior Analysis in-the-Wild (ABAW) competition.
