  Emotion semantic inconsistency is an ubiquitous challenge in multi-modal
sentiment analysis (MSA). MSA involves analyzing sentiment expressed across
various modalities like text, audio, and videos. Each modality may convey
distinct aspects of sentiment, due to subtle and nuanced expression of human
beings, leading to inconsistency, which may hinder the prediction of artificial
agents. In this work, we introduce a modality conflicting test set and assess
the performance of both traditional multi-modal sentiment analysis models and
multi-modal large language models (MLLMs). Our findings reveal significant
performance degradation across traditional models when confronted with
semantically conflicting data and point out the drawbacks of MLLMs when
handling multi-modal emotion analysis. Our research presents a new challenge
and offer valuable insights for the future development of sentiment analysis
systems.
