  Fake news detection has become a major task to solve as there has been an
increasing number of fake news on the internet in recent years. Although many
classification models have been proposed based on statistical learning methods
showing good results, reasoning behind the classification performances may not
be enough. In the self-supervised learning studies, it has been highlighted
that a quality of representation (embedding) space matters and directly affects
a downstream task performance. In this study, a quality of the representation
space is analyzed visually and analytically in terms of linear separability for
different classes on a real and fake news dataset. To further add
interpretability to a classification model, a modification of Class Activation
Mapping (CAM) is proposed. The modified CAM provides a CAM score for each word
token, where the CAM score on a word token denotes a level of focus on that
word token to make the prediction. Finally, it is shown that the naive BERT
model topped with a learnable linear layer is enough to achieve robust
performance while being compatible with CAM.
