  Social media studies often collect data retrospectively to analyze public
opinion. Social media data may decay over time and such decay may prevent the
collection of the complete dataset. As a result, the collected dataset may
differ from the complete dataset and the study may suffer from data persistence
bias. Past research suggests that the datasets collected retrospectively are
largely representative of the original dataset in terms of textual content.
However, no study analyzed the impact of data persistence bias on social media
studies such as those focusing on controversial topics. In this study, we
analyze the data persistence and the bias it introduces on the datasets of
three types: controversial topics, trending topics, and framing of issues. We
report which topics are more likely to suffer from data persistence among these
datasets. We quantify the data persistence bias using the change in political
orientation, the presence of potentially harmful content and topics as
measures. We found that controversial datasets are more likely to suffer from
data persistence and they lean towards the political left upon recollection.
The turnout of the data that contain potentially harmful content is
significantly lower on non-controversial datasets. Overall, we found that the
topics promoted by right-aligned users are more likely to suffer from data
persistence. Account suspensions are the primary factor contributing to data
removals, if not the only one. Our results emphasize the importance of
accounting for the data persistence bias by collecting the data in real time
when the dataset employed is vulnerable to data persistence bias.
