  Pre-training of neural networks has recently revolutionized the field of
Natural Language Processing (NLP) and has before demonstrated its effectiveness
in computer vision. At the same time, advances around the detection of fake
news were mainly driven by the context-based paradigm, where different types of
signals (e.g. from social media) form graph-like structures that hold
contextual information apart from the news article to classify. We propose to
merge these two developments by applying pre-training of Graph Neural Networks
(GNNs) in the domain of context-based fake news detection. Our experiments
provide an evaluation of different pre-training strategies for graph-based
misinformation detection and demonstrate that transfer learning does currently
not lead to significant improvements over training a model from scratch in the
domain. We argue that a major current issue is the lack of suitable large-scale
resources that can be used for pre-training.
