  Simplicity Bias (SB) is a phenomenon that deep neural networks tend to rely
favorably on simpler predictive patterns but ignore some complex features when
applied to supervised discriminative tasks. In this work, we investigate SB in
long-tailed image recognition and find the tail classes suffer more severely
from SB, which harms the generalization performance of such underrepresented
classes. We empirically report that self-supervised learning (SSL) can mitigate
SB and perform in complementary to the supervised counterpart by enriching the
features extracted from tail samples and consequently taking better advantage
of such rare samples. However, standard SSL methods are designed without
explicitly considering the inherent data distribution in terms of classes and
may not be optimal for long-tailed distributed data. To address this
limitation, we propose a novel SSL method tailored to imbalanced data. It
leverages SSL by triple diverse levels, i.e., holistic-, partial-, and
augmented-level, to enhance the learning of predictive complex patterns, which
provides the potential to overcome the severe SB on tail data. Both
quantitative and qualitative experimental results on five long-tailed benchmark
datasets show our method can effectively mitigate SB and significantly
outperform the competing state-of-the-arts.
