  We study the phenomenon of bias amplification in classifiers, wherein a
machine learning model learns to predict classes with a greater disparity than
the underlying ground truth. We demonstrate that bias amplification can arise
via an inductive bias in gradient descent methods that results in the
overestimation of the importance of moderately-predictive "weak" features if
insufficient training data is available. This overestimation gives rise to
feature-wise bias amplification -- a previously unreported form of bias that
can be traced back to the features of a trained model. Through analysis and
experiments, we show that while some bias cannot be mitigated without
sacrificing accuracy, feature-wise bias amplification can be mitigated through
targeted feature selection. We present two new feature selection algorithms for
mitigating bias amplification in linear models, and show how they can be
adapted to convolutional neural networks efficiently. Our experiments on
synthetic and real data demonstrate that these algorithms consistently lead to
reduced bias without harming accuracy, in some cases eliminating predictive
bias altogether while providing modest gains in accuracy.
