  Screening classifiers are increasingly used to identify qualified candidates
in a variety of selection processes. In this context, it has been recently
shown that, if a classifier is calibrated, one can identify the smallest set of
candidates which contains, in expectation, a desired number of qualified
candidates using a threshold decision rule. This lends support to focusing on
calibration as the only requirement for screening classifiers. In this paper,
we argue that screening policies that use calibrated classifiers may suffer
from an understudied type of within-group unfairness -- they may unfairly treat
qualified members within demographic groups of interest. Further, we argue that
this type of unfairness can be avoided if classifiers satisfy within-group
monotonicity, a natural monotonicity property within each of the groups. Then,
we introduce an efficient post-processing algorithm based on dynamic
programming to minimally modify a given calibrated classifier so that its
probability estimates satisfy within-group monotonicity. We validate our
algorithm using US Census survey data and show that within-group monotonicity
can be often achieved at a small cost in terms of prediction granularity and
shortlist size.
