  The increasingly common use of neural network classifiers in industrial and
social applications of image analysis has allowed impressive progress these
last years. Such methods are however sensitive to algorithmic bias, i.e. to an
under- or an over-representation of positive predictions or to higher
prediction errors in specific subgroups of images. We then introduce in this
paper a new method to temper the algorithmic bias in Neural-Network based
classifiers. Our method is Neural-Network architecture agnostic and scales well
to massive training sets of images. It indeed only overloads the loss function
with a Wasserstein-2 based regularization term for which we back-propagate the
impact of specific output predictions using a new model, based on the Gateaux
derivatives of the predictions distribution. This model is algorithmically
reasonable and makes it possible to use our regularized loss with standard
stochastic gradient-descent strategies. Its good behavior is assessed on the
reference Adult census, MNIST, CelebA datasets.
