  Ranking algorithms are being widely employed in various online hiring
platforms including LinkedIn, TaskRabbit, and Fiverr. Prior research has
demonstrated that ranking algorithms employed by these platforms are prone to a
variety of undesirable biases, leading to the proposal of fair ranking
algorithms (e.g., Det-Greedy) which increase exposure of underrepresented
candidates. However, there is little to no work that explores whether fair
ranking algorithms actually improve real world outcomes (e.g., hiring
decisions) for underrepresented groups. Furthermore, there is no clear
understanding as to how other factors (e.g., job context, inherent biases of
the employers) may impact the efficacy of fair ranking in practice. In this
work, we analyze various sources of gender biases in online hiring platforms,
including the job context and inherent biases of employers and establish how
these factors interact with ranking algorithms to affect hiring decisions. To
the best of our knowledge, this work makes the first attempt at studying the
interplay between the aforementioned factors in the context of online hiring.
We carry out a largescale user study simulating online hiring scenarios with
data from TaskRabbit, a popular online freelancing site. Our results
demonstrate that while fair ranking algorithms generally improve the selection
rates of underrepresented minorities, their effectiveness relies heavily on the
job contexts and candidate profiles.
