  A crucial aspect of a rumor detection model is its ability to generalize,
particularly its ability to detect emerging, previously unknown rumors. Past
research has indicated that content-based (i.e., using solely source posts as
input) rumor detection models tend to perform less effectively on unseen
rumors. At the same time, the potential of context-based models remains largely
untapped. The main contribution of this paper is in the in-depth evaluation of
the performance gap between content and context-based models specifically on
detecting new, unseen rumors. Our empirical findings demonstrate that
context-based models are still overly dependent on the information derived from
the rumors' source post and tend to overlook the significant role that
contextual information can play. We also study the effect of data split
strategies on classifier performance. Based on our experimental results, the
paper also offers practical suggestions on how to minimize the effects of
temporal concept drift in static datasets during the training of rumor
detection methods.
