  In consequential decision-making applications, mitigating unwanted biases in
machine learning models that yield systematic disadvantage to members of groups
delineated by sensitive attributes such as race and gender is one key
intervention to strive for equity. Focusing on demographic parity and equality
of opportunity, in this paper we propose an algorithm that improves the
fairness of a pre-trained classifier by simply dropping carefully selected
training data points. We select instances based on their influence on the
fairness metric of interest, computed using an infinitesimal jackknife-based
approach. The dropping of training points is done in principle, but in practice
does not require the model to be refit. Crucially, we find that such an
intervention does not substantially reduce the predictive performance of the
model but drastically improves the fairness metric. Through careful
experiments, we evaluate the effectiveness of the proposed approach on diverse
tasks and find that it consistently improves upon existing alternatives.
