  A limited amount of studies investigates the role of model-agnostic
adversarial behavior in toxic content classification. As toxicity classifiers
predominantly rely on lexical cues, (deliberately) creative and evolving
language-use can be detrimental to the utility of current corpora and
state-of-the-art models when they are deployed for content moderation. The less
training data is available, the more vulnerable models might become. This study
is, to our knowledge, the first to investigate the effect of adversarial
behavior and augmentation for cyberbullying detection. We demonstrate that
model-agnostic lexical substitutions significantly hurt classifier performance.
Moreover, when these perturbed samples are used for augmentation, we show
models become robust against word-level perturbations at a slight trade-off in
overall task performance. Augmentations proposed in prior work on toxicity
prove to be less effective. Our results underline the need for such evaluations
in online harm areas with small corpora. The perturbed data, models, and code
are available for reproduction at https://github.com/cmry/augtox
