  Long-tailed datasets, where head classes comprise much more training samples
than tail classes, cause recognition models to get biased towards the head
classes. Weighted loss is one of the most popular ways of mitigating this
issue, and a recent work has suggested that class-difficulty might be a better
clue than conventionally used class-frequency to decide the distribution of
weights. A heuristic formulation was used in the previous work for quantifying
the difficulty, but we empirically find that the optimal formulation varies
depending on the characteristics of datasets. Therefore, we propose
Difficulty-Net, which learns to predict the difficulty of classes using the
model's performance in a meta-learning framework. To make it learn reasonable
difficulty of a class within the context of other classes, we newly introduce
two key concepts, namely the relative difficulty and the driver loss. The
former helps Difficulty-Net take other classes into account when calculating
difficulty of a class, while the latter is indispensable for guiding the
learning to a meaningful direction. Extensive experiments on popular
long-tailed datasets demonstrated the effectiveness of the proposed method, and
it achieved state-of-the-art performance on multiple long-tailed datasets.
