  Designing an effective representation learning method for multimodal
sentiment analysis tasks is a crucial research direction. The challenge lies in
learning both shared and private information in a complete modal
representation, which is difficult with uniform multimodal labels and a raw
feature fusion approach. In this work, we propose a deep modal shared
information learning module based on the covariance matrix to capture the
shared information between modalities. Additionally, we use a label generation
module based on a self-supervised learning strategy to capture the private
information of the modalities. Our module is plug-and-play in multimodal tasks,
and by changing the parameterization, it can adjust the information exchange
relationship between the modes and learn the private or shared information
between the specified modes. We also employ a multi-task learning strategy to
help the model focus its attention on the modal differentiation training data.
We provide a detailed formulation derivation and feasibility proof for the
design of the deep modal shared information learning module. We conduct
extensive experiments on three common multimodal sentiment analysis baseline
datasets, and the experimental results validate the reliability of our model.
Furthermore, we explore more combinatorial techniques for the use of the
module. Our approach outperforms current state-of-the-art methods on most of
the metrics of the three public datasets.
