  Knowledge distillation has been applied to image classification successfully.
However, object detection is much more sophisticated and most knowledge
distillation methods have failed on it. In this paper, we point out that in
object detection, the features of the teacher and student vary greatly in
different areas, especially in the foreground and background. If we distill
them equally, the uneven differences between feature maps will negatively
affect the distillation. Thus, we propose Focal and Global Distillation (FGD).
Focal distillation separates the foreground and background, forcing the student
to focus on the teacher's critical pixels and channels. Global distillation
rebuilds the relation between different pixels and transfers it from teachers
to students, compensating for missing global information in focal distillation.
As our method only needs to calculate the loss on the feature map, FGD can be
applied to various detectors. We experiment on various detectors with different
backbones and the results show that the student detector achieves excellent mAP
improvement. For example, ResNet-50 based RetinaNet, Faster RCNN, RepPoints and
Mask RCNN with our distillation method achieve 40.7%, 42.0%, 42.0% and 42.1%
mAP on COCO2017, which are 3.3, 3.6, 3.4 and 2.9 higher than the baseline,
respectively. Our codes are available at https://github.com/yzd-v/FGD.
