  Recently proposed decoupled training methods emerge as a dominant paradigm
for long-tailed object detection. But they require an extra fine-tuning stage,
and the disjointed optimization of representation and classifier might lead to
suboptimal results. However, end-to-end training methods, like equalization
loss (EQL), still perform worse than decoupled training methods. In this paper,
we reveal the main issue in long-tailed object detection is the imbalanced
gradients between positives and negatives, and find that EQL does not solve it
well. To address the problem of imbalanced gradients, we introduce a new
version of equalization loss, called equalization loss v2 (EQL v2), a novel
gradient guided reweighing mechanism that re-balances the training process for
each category independently and equally. Extensive experiments are performed on
the challenging LVIS benchmark. EQL v2 outperforms origin EQL by about 4 points
overall AP with 14-18 points improvements on the rare categories. More
importantly, it also surpasses decoupled training methods. Without further
tuning for the Open Images dataset, EQL v2 improves EQL by 7.3 points AP,
showing strong generalization ability. Codes have been released at
https://github.com/tztztztztz/eqlv2
