  Effective machine learning models can automatically learn useful information
from a large quantity of data and provide decisions in a high accuracy. These
models may, however, lead to unfair predictions in certain sense among the
population groups of interest, where the grouping is based on such sensitive
attributes as race and gender. Various fairness definitions, such as
demographic parity and equalized odds, were proposed in prior art to ensure
that decisions guided by the machine learning models are equitable.
Unfortunately, the "fair" model trained with these fairness definitions is
threshold sensitive, i.e., the condition of fairness may no longer hold true
when tuning the decision threshold. This paper introduces the notion of
threshold invariant fairness, which enforces equitable performances across
different groups independent of the decision threshold. To achieve this goal,
this paper proposes to equalize the risk distributions among the groups via two
approximation methods. Experimental results demonstrate that the proposed
methodology is effective to alleviate the threshold sensitivity in machine
learning models designed to achieve fairness.
