  Developing a system to detect online offensive language is very important to
the health and the security of online users. Studies have shown that cyberhate,
online harassment and other misuses of technology are on the rise, particularly
during the global Coronavirus pandemic in 2020. According to the latest report
by the Anti-Defamation League (ADL), 35% of online users reported online
harassment related to their identity-based characteristics, which is a 3%
increase over 2019. Applying advanced techniques from the Natural Language
Processing (NLP) field to support the development of an online hate-free
community is a critical task for social justice. Transfer learning enhances the
performance of the classifier by allowing the transfer of knowledge from one
domain or one dataset to others that have not been seen before, thus,
supporting the classifier to be more generalizable. In our study, we apply the
principles of transfer learning cross multiple Arabic offensive language
datasets to compare the effects on system performance. This study aims at
investigating the effects of fine-tuning and training Bidirectional Encoder
Representations from Transformers (BERT) model on multiple Arabic offensive
language datasets individually and testing it using other datasets
individually. Our experiment starts with a comparison among multiple BERT
models to guide the selection of the main model that is used for our study. The
study also investigates the effects of concatenating all datasets to be used
for fine-tuning and training BERT model. Our results demonstrate the limited
effects of transfer learning on the performance of the classifiers,
particularly for highly dialectic comments.
