  As a general model compression paradigm, feature-based knowledge distillation
allows the student model to learn expressive features from the teacher
counterpart. In this paper, we mainly focus on designing an effective
feature-distillation framework and propose a spatial-channel adaptive masked
distillation (AMD) network for object detection. More specifically, in order to
accurately reconstruct important feature regions, we first perform
attention-guided feature masking on the feature map of the student network,
such that we can identify the important features via spatially adaptive feature
masking instead of random masking in the previous methods. In addition, we
employ a simple and efficient module to allow the student network channel to be
adaptive, improving its model capability in object perception and detection. In
contrast to the previous methods, more crucial object-aware features can be
reconstructed and learned from the proposed network, which is conducive to
accurate object detection. The empirical experiments demonstrate the
superiority of our method: with the help of our proposed distillation method,
the student networks report 41.3%, 42.4%, and 42.7% mAP scores when RetinaNet,
Cascade Mask-RCNN and RepPoints are respectively used as the teacher framework
for object detection, which outperforms the previous state-of-the-art
distillation methods including FGD and MGD.
