  Learning generalizable representation and classifier for class-imbalanced
data is challenging for data-driven deep models. Most studies attempt to
re-balance the data distribution, which is prone to overfitting on tail classes
and underfitting on head classes. In this work, we propose Dual Compensation
Residual Networks to better fit both tail and head classes. Firstly, we propose
dual Feature Compensation Module (FCM) and Logit Compensation Module (LCM) to
alleviate the overfitting issue. The design of these two modules is based on
the observation: an important factor causing overfitting is that there is
severe feature drift between training and test data on tail classes. In
details, the test features of a tail category tend to drift towards feature
cloud of multiple similar head categories. So FCM estimates a multi-mode
feature drift direction for each tail category and compensate for it.
Furthermore, LCM translates the deterministic feature drift vector estimated by
FCM along intra-class variations, so as to cover a larger effective
compensation space, thereby better fitting the test features. Secondly, we
propose a Residual Balanced Multi-Proxies Classifier (RBMC) to alleviate the
under-fitting issue. Motivated by the observation that re-balancing strategy
hinders the classifier from learning sufficient head knowledge and eventually
causes underfitting, RBMC utilizes uniform learning with a residual path to
facilitate classifier learning. Comprehensive experiments on Long-tailed and
Class-Incremental benchmarks validate the efficacy of our method.
