  Social media misinformation harms individuals and societies and is
potentialized by fast-growing multi-modal content (i.e., texts and images),
which accounts for higher "credibility" than text-only news pieces. Although
existing supervised misinformation detection methods have obtained acceptable
performances in key setups, they may require large amounts of labeled data from
various events, which can be time-consuming and tedious. In turn, directly
training a model by leveraging a publicly available dataset may fail to
generalize due to domain shifts between the training data (a.k.a. source
domains) and the data from target domains. Most prior work on domain shift
focuses on a single modality (e.g., text modality) and ignores the scenario
where sufficient unlabeled target domain data may not be readily available in
an early stage. The lack of data often happens due to the dynamic propagation
trend (i.e., the number of posts related to fake news increases slowly before
catching the public attention). We propose a novel robust domain and
cross-modal approach (\textbf{RDCM}) for multi-modal misinformation detection.
It reduces the domain shift by aligning the joint distribution of textual and
visual modalities through an inter-domain alignment module and bridges the
semantic gap between both modalities through a cross-modality alignment module.
We also propose a framework that simultaneously considers application scenarios
of domain generalization (in which the target domain data is unavailable) and
domain adaptation (in which unlabeled target domain data is available).
Evaluation results on two public multi-modal misinformation detection datasets
(Pheme and Twitter Datasets) evince the superiority of the proposed model. The
formal implementation of this paper can be found in this link:
https://github.com/less-and-less-bugs/RDCM
