  Perception of offensiveness is inherently subjective, shaped by the lived
experiences and socio-cultural values of the perceivers. Recent years have seen
substantial efforts to build AI-based tools that can detect offensive language
at scale, as a means to moderate social media platforms, and to ensure safety
of conversational AI technologies such as ChatGPT and Bard. However, existing
approaches treat this task as a technical endeavor, built on top of data
annotated for offensiveness by a global crowd workforce without any attention
to the crowd workers' provenance or the values their perceptions reflect. We
argue that cultural and psychological factors play a vital role in the
cognitive processing of offensiveness, which is critical to consider in this
context. We re-frame the task of determining offensiveness as essentially a
matter of moral judgment -- deciding the boundaries of ethically wrong vs.
right language within an implied set of socio-cultural norms. Through a
large-scale cross-cultural study based on 4309 participants from 21 countries
across 8 cultural regions, we demonstrate substantial cross-cultural
differences in perceptions of offensiveness. More importantly, we find that
individual moral values play a crucial role in shaping these variations: moral
concerns about Care and Purity are significant mediating factors driving
cross-cultural differences. These insights are of crucial importance as we
build AI models for the pluralistic world, where the values they espouse should
aim to respect and account for moral values in diverse geo-cultural contexts.
