  Knowledge distillation (KD) has witnessed its powerful capability in learning
compact models in object detection. Previous KD methods for object detection
mostly focus on imitating deep features within the imitation regions instead of
mimicking classification logit due to its inefficiency in distilling
localization information and trivial improvement. In this paper, by
reformulating the knowledge distillation process on localization, we present a
novel localization distillation (LD) method which can efficiently transfer the
localization knowledge from the teacher to the student. Moreover, we also
heuristically introduce the concept of valuable localization region that can
aid to selectively distill the semantic and localization knowledge for a
certain region. Combining these two new components, for the first time, we show
that logit mimicking can outperform feature imitation and localization
knowledge distillation is more important and efficient than semantic knowledge
for distilling object detectors. Our distillation scheme is simple as well as
effective and can be easily applied to different dense object detectors.
Experiments show that our LD can boost the AP score of GFocal-ResNet-50 with a
single-scale 1x training schedule from 40.1 to 42.1 on the COCO benchmark
without any sacrifice on the inference speed. Our source code and trained
models are publicly available at https://github.com/HikariTJU/LD
