  Personal attacks in the context of social media conversations often lead to
fast-paced derailment, leading to even more harmful exchanges being made.
State-of-the-art systems for the detection of such conversational derailment
often make use of deep learning approaches for prediction purposes. In this
paper, we show that an Attention-based BERT architecture, pre-trained on a
large Twitter corpus and fine-tuned on our task, is efficient and effective in
making such predictions. This model shows clear advantages in performance to
the existing LSTM model we use as a baseline. Additionally, we show that this
impressive performance can be attained through fine-tuning on a relatively
small, novel dataset, particularly after mitigating overfitting issues through
synthetic oversampling techniques. By introducing the first transformer based
model for forecasting conversational events on Twitter, this work lays the
foundation for a practical tool to encourage better interactions on one of the
most ubiquitous social media platforms.
