  Many important decisions in societies such as school admissions, hiring, or
elections are based on the selection of top-ranking individuals from a larger
pool of candidates. This process is often subject to biases, which typically
manifest as an under-representation of certain groups among the selected or
accepted individuals. The most common approach to this issue is debiasing, for
example via the introduction of quotas that ensure proportional representation
of groups with respect to a certain, often binary attribute. Cases include
quotas for women on corporate boards or ethnic quotas in elections. This,
however, has the potential to induce changes in representation with respect to
other attributes. For the case of two correlated binary attributes we show that
quota-based debiasing based on a single attribute can worsen the representation
of already underrepresented groups and decrease overall fairness of selection.
We use several data sets from a broad range of domains from recidivism risk
assessments to scientific citations to assess this effect in real-world
settings. Our results demonstrate the importance of including all relevant
attributes in debiasing procedures and that more efforts need to be put into
eliminating the root causes of inequalities as purely numerical solutions such
as quota-based debiasing might lead to unintended consequences.
