  Hate speech and profanity detection suffer from data sparsity, especially for
languages other than English, due to the subjective nature of the tasks and the
resulting annotation incompatibility of existing corpora. In this study, we
identify profane subspaces in word and sentence representations and explore
their generalization capability on a variety of similar and distant target
tasks in a zero-shot setting. This is done monolingually (German) and
cross-lingually to closely-related (English), distantly-related (French) and
non-related (Arabic) tasks. We observe that, on both similar and distant target
tasks and across all languages, the subspace-based representations transfer
more effectively than standard BERT representations in the zero-shot setting,
with improvements between F1 +10.9 and F1 +42.9 over the baselines across all
tested monolingual and cross-lingual scenarios.
