  Weighting strategy prevails in machine learning. For example, a common
approach in robust machine learning is to exert lower weights on samples which
are likely to be noisy or quite hard. This study reveals another undiscovered
strategy, namely, compensating. Various incarnations of compensating have been
utilized but it has not been explicitly revealed. Learning with compensating is
called compensation learning and a systematic taxonomy is constructed for it in
this study. In our taxonomy, compensation learning is divided on the basis of
the compensation targets, directions, inference manners, and granularity
levels. Many existing learning algorithms including some classical ones can be
viewed or understood at least partially as compensation techniques.
Furthermore, a family of new learning algorithms can be obtained by plugging
the compensation learning into existing learning algorithms. Specifically, two
concrete new learning algorithms are proposed for robust machine learning.
Extensive experiments on image classification and text sentiment analysis
verify the effectiveness of the two new algorithms. Compensation learning can
also be used in other various learning scenarios, such as imbalance learning,
clustering, regression, and so on.
