  Multimodal sentiment analysis benefits various applications such as
human-computer interaction and recommendation systems. It aims to infer the
users' bipolar ideas using visual, textual, and acoustic signals. Although
researchers affirm the association between cognitive cues and emotional
manifestations, most of the current multimodal approaches in sentiment analysis
disregard user-specific aspects. To tackle this issue, we devise a novel method
to perform multimodal sentiment prediction using cognitive cues, such as
personality. Our framework constructs an adaptive tree by hierarchically
dividing users and trains the LSTM-based submodels, utilizing an
attention-based fusion to transfer cognitive-oriented knowledge within the
tree. Subsequently, the framework consumes the conclusive agglomerative
knowledge from the adaptive tree to predict final sentiments. We also devise a
dynamic dropout method to facilitate data sharing between neighboring nodes,
reducing data sparsity. The empirical results on real-world datasets determine
that our proposed model for sentiment prediction can surpass trending rivals.
Moreover, compared to other ensemble approaches, the proposed transfer-based
algorithm can better utilize the latent cognitive cues and foster the
prediction outcomes. Based on the given extrinsic and intrinsic analysis
results, we note that compared to other theoretical-based techniques, the
proposed hierarchical clustering approach can better group the users within the
adaptive tree.
