  We investigate different strategies for automatic offensive language
classification on German Twitter data. For this, we employ a sequentially
combined BiLSTM-CNN neural network. Based on this model, three transfer
learning tasks to improve the classification performance with background
knowledge are tested. We compare 1. Supervised category transfer: social media
data annotated with near-offensive language categories, 2. Weakly-supervised
category transfer: tweets annotated with emojis they contain, 3. Unsupervised
category transfer: tweets annotated with topic clusters obtained by Latent
Dirichlet Allocation (LDA). Further, we investigate the effect of three
different strategies to mitigate negative effects of 'catastrophic forgetting'
during transfer learning. Our results indicate that transfer learning in
general improves offensive language detection. Best results are achieved from
pre-training our model on the unsupervised topic clustering of tweets in
combination with thematic user cluster information.
