  Conceptual knowledge is fundamental to human cognition and knowledge bases.
However, existing knowledge probing works only focus on evaluating factual
knowledge of pre-trained language models (PLMs) and ignore conceptual
knowledge. Since conceptual knowledge often appears as implicit commonsense
behind texts, designing probes for conceptual knowledge is hard. Inspired by
knowledge representation schemata, we comprehensively evaluate conceptual
knowledge of PLMs by designing three tasks to probe whether PLMs organize
entities by conceptual similarities, learn conceptual properties, and
conceptualize entities in contexts, respectively. For the tasks, we collect and
annotate 24k data instances covering 393 concepts, which is COPEN, a COnceptual
knowledge Probing bENchmark. Extensive experiments on different sizes and types
of PLMs show that existing PLMs systematically lack conceptual knowledge and
suffer from various spurious correlations. We believe this is a critical
bottleneck for realizing human-like cognition in PLMs. COPEN and our codes are
publicly released at https://github.com/THU-KEG/COPEN.
