  Offensive language detection is increasingly crucial for maintaining a
civilized social media platform and deploying pre-trained language models.
However, this task in Chinese is still under exploration due to the scarcity of
reliable datasets. To this end, we propose a benchmark --COLD for Chinese
offensive language analysis, including a Chinese Offensive Language Dataset
--COLDATASET and a baseline detector --COLDETECTOR which is trained on the
dataset. We show that the COLD benchmark contributes to Chinese offensive
language detection which is challenging for existing resources. We then deploy
the COLDETECTOR and conduct detailed analyses on popular Chinese pre-trained
language models. We first analyze the offensiveness of existing generative
models and show that these models inevitably expose varying degrees of
offensive issues. Furthermore, we investigate the factors that influence the
offensive generations, and we find that anti-bias contents and keywords
referring to certain groups or revealing negative attitudes trigger offensive
outputs easier.
