  Utilizing state-of-the-art Large Language Models (LLMs), automatic code
generation models play a pivotal role in enhancing the productivity of software
development procedures. As the adoption of LLMs becomes more widespread in
software coding ecosystems, a pressing issue has emerged: does the generated
code contain social bias and unfairness, such as those related to age, gender,
and race? This issue concerns the integrity, fairness, and ethical foundation
of software applications that depend on the code generated by these models, yet
is under-explored in the literature. This paper presents a novel bias testing
framework that is specifically designed for code generation tasks. Based on
this framework, we conduct an extensive evaluation of the bias in code
generated by five state-of-the-art LLMs. Our findings reveal that 20.29% to
44.93% code functions generated by the models under study are biased when
handling bias sensitive tasks (i.e., tasks that involve sensitive attributes
such as age and gender). This indicates that the existing LLMs can be unfair in
code generation, posing risks of unintended and harmful software behaviors. To
mitigate bias for code generation models, we evaluate five bias mitigation
prompt strategies, i.e., utilizing bias testing results to refine the code
(zero-shot), one-, few-shot, and two Chain-of-Thought (CoT) prompts. Our
evaluation results illustrate that these strategies are all effective in
mitigating bias. Overall, one-shot and few-shot learning are the two most
effective. For GPT-4, 80% to 90% code bias can be removed with one-shot
learning.
