  Knowledge distillation (KD) is a widely adopted and effective method for
compressing models in object detection tasks. Particularly, feature-based
distillation methods have shown remarkable performance. Existing approaches
often ignore the uncertainty in the teacher model's knowledge, which stems from
data noise and imperfect training. This limits the student model's ability to
learn latent knowledge, as it may overly rely on the teacher's imperfect
guidance. In this paper, we propose a novel feature-based distillation paradigm
with knowledge uncertainty for object detection, termed "Uncertainty
Estimation-Discriminative Knowledge Extraction-Knowledge Transfer (UET)", which
can seamlessly integrate with existing distillation methods. By leveraging the
Monte Carlo dropout technique, we introduce knowledge uncertainty into the
training process of the student model, facilitating deeper exploration of
latent knowledge. Our method performs effectively during the KD process without
requiring intricate structures or extensive computational resources. Extensive
experiments validate the effectiveness of our proposed approach across various
distillation strategies, detectors, and backbone architectures. Specifically,
following our proposed paradigm, the existing FGD method achieves
state-of-the-art (SoTA) performance, with ResNet50-based GFL achieving 44.1%
mAP on the COCO dataset, surpassing the baselines by 3.9%.
