  In this work, we propose to progressively increase the training difficulty
during learning a neural network model via a novel strategy which we call
mini-batch trimming. This strategy makes sure that the optimizer puts its focus
in the later training stages on the more difficult samples, which we identify
as the ones with the highest loss in the current mini-batch. The strategy is
very easy to integrate into an existing training pipeline and does not
necessitate a change of the network model. Experiments on several image
classification problems show that mini-batch trimming is able to increase the
generalization ability (measured via final test error) of the trained model.
