  With the popularity of automatic code generation tools, such as Copilot, the
study of the potential hazards of these tools is gaining importance. In this
work, we explore the social bias problem in pre-trained code generation models.
We propose a new paradigm to construct code prompts and successfully uncover
social biases in code generation models. To quantify the severity of social
biases in generated code, we develop a dataset along with three metrics to
evaluate the overall social bias and fine-grained unfairness across different
demographics. Experimental results on three pre-trained code generation models
(Codex, InCoder, and CodeGen) with varying sizes, reveal severe social biases.
Moreover, we conduct analysis to provide useful insights for further choice of
code generation models with low social bias. (This work contains examples that
potentially implicate stereotypes, associations, and other harms that could be
offensive to individuals in certain social groups.)
