  The standard paradigm for fake news detection mainly utilizes text
information to model the truthfulness of news. However, the discourse of online
fake news is typically subtle and it requires expert knowledge to use textual
information to debunk fake news. Recently, studies focusing on multimodal fake
news detection have outperformed text-only methods. Recent approaches utilizing
the pre-trained model to extract unimodal features, or fine-tuning the
pre-trained model directly, have become a new paradigm for detecting fake news.
Again, this paradigm either requires a large number of training instances, or
updates the entire set of pre-trained model parameters, making real-world fake
news detection impractical. Furthermore, traditional multimodal methods fuse
the cross-modal features directly without considering that the uncorrelated
semantic representation might inject noise into the multimodal features. This
paper proposes a Similarity-Aware Multimodal Prompt Learning (SAMPLE)
framework. First, we incorporate prompt learning into multimodal fake news
detection. Prompt learning, which only tunes prompts with a frozen language
model, can reduce memory usage significantly and achieve comparable
performances, compared with fine-tuning. We analyse three prompt templates with
a soft verbalizer to detect fake news. In addition, we introduce the
similarity-aware fusing method to adaptively fuse the intensity of multimodal
representation and mitigate the noise injection via uncorrelated cross-modal
features. For evaluation, SAMPLE surpasses the F1 and the accuracies of
previous works on two benchmark multimodal datasets, demonstrating the
effectiveness of the proposed method in detecting fake news. In addition,
SAMPLE also is superior to other approaches regardless of few-shot and
data-rich settings.
