  Most of object detection algorithms can be categorized into two classes:
two-stage detectors and one-stage detectors. Recently, many efforts have been
devoted to one-stage detectors for the simple yet effective architecture.
Different from two-stage detectors, one-stage detectors aim to identify
foreground objects from all candidates in a single stage. This architecture is
efficient but can suffer from the imbalance issue with respect to two aspects:
the inter-class imbalance between the number of candidates from foreground and
background classes and the intra-class imbalance in the hardness of background
candidates, where only a few candidates are hard to be identified. In this
work, we propose a novel distributional ranking (DR) loss to handle the
challenge. For each image, we convert the classification problem to a ranking
problem, which considers pairs of candidates within the image, to address the
inter-class imbalance problem. Then, we push the distributions of confidence
scores for foreground and background towards the decision boundary. After that,
we optimize the rank of the expectations of derived distributions in lieu of
original pairs. Our method not only mitigates the intra-class imbalance issue
in background candidates but also improves the efficiency for the ranking
algorithm. By merely replacing the focal loss in RetinaNet with the developed
DR loss and applying ResNet-101 as the backbone, mAP of the single-scale test
on COCO can be improved from 39.1% to 41.7% without bells and whistles, which
demonstrates the effectiveness of the proposed loss function. Code is available
at \url{https://github.com/idstcv/DR_loss}.
