  The fairness of machine learning (ML) approaches is critical to the
reliability of modern artificial intelligence systems. Despite extensive study
on this topic, the fairness of ML models in the software engineering (SE)
domain has not been well explored yet. As a result, many ML-powered software
systems, particularly those utilized in the software engineering community,
continue to be prone to fairness issues. Taking one of the typical SE tasks,
i.e., code reviewer recommendation, as a subject, this paper conducts the first
study toward investigating the issue of fairness of ML applications in the SE
domain. Our empirical study demonstrates that current state-of-the-art ML-based
code reviewer recommendation techniques exhibit unfairness and discriminating
behaviors. Specifically, male reviewers get on average 7.25% more
recommendations than female code reviewers compared to their distribution in
the reviewer set. This paper also discusses the reasons why the studied
ML-based code reviewer recommendation systems are unfair and provides solutions
to mitigate the unfairness. Our study further indicates that the existing
mitigation methods can enhance fairness by 100% in projects with a similar
distribution of protected and privileged groups, but their effectiveness in
improving fairness on imbalanced or skewed data is limited. Eventually, we
suggest a solution to overcome the drawbacks of existing mitigation techniques
and tackle bias in datasets that are imbalanced or skewed.
