  Data in the real world tends to exhibit a long-tailed label distribution,
which poses great challenges for the training of neural networks in visual
recognition. Existing methods tackle this problem mainly from the perspective
of data quantity, i.e., the number of samples in each class. To be specific,
they pay more attention to tail classes, like applying larger adjustments to
the logit. However, in the training process, the quantity and difficulty of
data are two intertwined and equally crucial problems. For some tail classes,
the features of their instances are distinct and discriminative, which can also
bring satisfactory accuracy; for some head classes, although with sufficient
samples, the high semantic similarity with other classes and lack of
discriminative features will bring bad accuracy. Based on these observations,
we propose Adaptive Logit Adjustment Loss (ALA Loss) to apply an adaptive
adjusting term to the logit. The adaptive adjusting term is composed of two
complementary factors: 1) quantity factor, which pays more attention to tail
classes, and 2) difficulty factor, which adaptively pays more attention to hard
instances in the training process. The difficulty factor can alleviate the
over-optimization on tail yet easy instances and under-optimization on head yet
hard instances. The synergy of the two factors can not only advance the
performance on tail classes even further, but also promote the accuracy on head
classes. Unlike previous logit adjusting methods that only concerned about data
quantity, ALA Loss tackles the long-tailed problem from a more comprehensive,
fine-grained and adaptive perspective. Extensive experimental results show that
our method achieves the state-of-the-art performance on challenging recognition
benchmarks, including ImageNet-LT, iNaturalist 2018, and Places-LT.
