  Classification and regression are two pillars of object detectors. In most
CNN-based detectors, these two pillars are optimized independently. Without
direct interactions between them, the classification loss and the regression
loss can not be optimized synchronously toward the optimal direction in the
training phase. This clearly leads to lots of inconsistent predictions with
high classification score but low localization accuracy or low classification
score but high localization accuracy in the inference phase, especially for the
objects of irregular shape and occlusion, which severely hurts the detection
performance of existing detectors after NMS. To reconcile prediction
consistency for balanced object detection, we propose a Harmonic loss to
harmonize the optimization of classification branch and localization branch.
The Harmonic loss enables these two branches to supervise and promote each
other during training, thereby producing consistent predictions with high
co-occurrence of top classification and localization in the inference phase.
Furthermore, in order to prevent the localization loss from being dominated by
outliers during training phase, a Harmonic IoU loss is proposed to harmonize
the weight of the localization loss of different IoU-level samples.
Comprehensive experiments on benchmarks PASCAL VOC and MS COCO demonstrate the
generality and effectiveness of our model for facilitating existing object
detectors to state-of-the-art accuracy.
