  Multimodal emotion recognition in conversation (MERC) has garnered
substantial research attention recently. Existing MERC methods face several
challenges: (1) they fail to fully harness direct inter-modal cues, possibly
leading to less-than-thorough cross-modal modeling; (2) they concurrently
extract information from the same and different modalities at each network
layer, potentially triggering conflicts from the fusion of multi-source data;
(3) they lack the agility required to detect dynamic sentimental changes,
perhaps resulting in inaccurate classification of utterances with abrupt
sentiment shifts. To address these issues, a novel approach named GraphSmile is
proposed for tracking intricate emotional cues in multimodal dialogues.
GraphSmile comprises two key components, i.e., GSF and SDP modules. GSF
ingeniously leverages graph structures to alternately assimilate inter-modal
and intra-modal emotional dependencies layer by layer, adequately capturing
cross-modal cues while effectively circumventing fusion conflicts. SDP is an
auxiliary task to explicitly delineate the sentiment dynamics between
utterances, promoting the model's ability to distinguish sentimental
discrepancies. Furthermore, GraphSmile is effortlessly applied to multimodal
sentiment analysis in conversation (MSAC), forging a unified multimodal
affective model capable of executing MERC and MSAC tasks. Empirical results on
multiple benchmarks demonstrate that GraphSmile can handle complex emotional
and sentimental patterns, significantly outperforming baseline models.
