  Long-tailed class distributions are prevalent among the practical
applications of object detection and instance segmentation. Prior work in
long-tail instance segmentation addresses the imbalance of losses between rare
and frequent categories by reducing the penalty for a model incorrectly
predicting a rare class label. We demonstrate that the rare categories are
heavily suppressed by correct background predictions, which reduces the
probability for all foreground categories with equal weight. Due to the
relative infrequency of rare categories, this leads to an imbalance that biases
towards predicting more frequent categories. Based on this insight, we develop
DropLoss -- a novel adaptive loss to compensate for this imbalance without a
trade-off between rare and frequent categories. With this loss, we show
state-of-the-art mAP across rare, common, and frequent categories on the LVIS
dataset.
