  Hate speech is an important problem in the management of user-generated
content. To remove offensive content or ban misbehaving users, content
moderators need reliable hate speech detectors. Recently, deep neural networks
based on the transformer architecture, such as the (multilingual) BERT model,
achieve superior performance in many natural language classification tasks,
including hate speech detection. So far, these methods have not been able to
quantify their output in terms of reliability. We propose a Bayesian method
using Monte Carlo dropout within the attention layers of the transformer models
to provide well-calibrated reliability estimates. We evaluate and visualize the
results of the proposed approach on hate speech detection problems in several
languages. Additionally, we test if affective dimensions can enhance the
information extracted by the BERT model in hate speech classification. Our
experiments show that Monte Carlo dropout provides a viable mechanism for
reliability estimation in transformer networks. Used within the BERT model, it
ofers state-of-the-art classification performance and can detect less trusted
predictions. Also, it was observed that affective dimensions extracted using
sentic computing methods can provide insights toward interpretation of emotions
involved in hate speech. Our approach not only improves the classification
performance of the state-of-the-art multilingual BERT model but the computed
reliability scores also significantly reduce the workload in an inspection of
ofending cases and reannotation campaigns. The provided visualization helps to
understand the borderline outcomes.
