  Current content moderation practices follow the trial-and-error approach,
meaning that moderators apply sequences of interventions until they obtain the
desired outcome. However, being able to preemptively estimate the effects of an
intervention would allow moderators the unprecedented opportunity to plan their
actions ahead of application. As a first step towards this goal, here we
propose and tackle the novel task of predicting the effect of a moderation
intervention. We study the reactions of 16,540 users to a massive ban of online
communities on Reddit, training a set of binary classifiers to identify those
users who would abandon the platform after the intervention - a problem of
great practical relevance. We leverage a dataset of 13.8M posts to compute a
large and diverse set of 142 features, which convey information about the
activity, toxicity, relations, and writing style of the users. We obtain
promising results, with the best-performing model achieving micro F1 = 0.800
and macro F1 = 0.676. Our model demonstrates robust generalizability when
applied to users from previously unseen communities. Furthermore, we identify
activity features as the most informative predictors, followed by relational
and toxicity features, while writing style features exhibit limited utility.
Our results demonstrate the feasibility of predicting the effects of a
moderation intervention, paving the way for a new research direction in
predictive content moderation aimed at empowering moderators with intelligent
tools to plan ahead their actions.
