  Benchmark datasets for visual recognition assume that data is uniformly
distributed, while real-world datasets obey long-tailed distribution. Current
approaches handle the long-tailed problem to transform the long-tailed dataset
to uniform distribution by re-sampling or re-weighting strategies. These
approaches emphasize the tail classes but ignore the hard examples in head
classes, which result in performance degradation. In this paper, we propose a
novel gradient harmonized mechanism with category-wise adaptive precision to
decouple the difficulty and sample size imbalance in the long-tailed problem,
which are correspondingly solved via intra- and inter-category balance
strategies. Specifically, intra-category balance focuses on the hard examples
in each category to optimize the decision boundary, while inter-category
balance aims to correct the shift of decision boundary by taking each category
as a unit. Extensive experiments demonstrate that the proposed method
consistently outperforms other approaches on all the datasets.
