  In this paper, we present a novel training scheme, namely Teach-DETR, to
learn better DETR-based detectors from versatile teacher detectors. We show
that the predicted boxes from teacher detectors are effective medium to
transfer knowledge of teacher detectors, which could be either RCNN-based or
DETR-based detectors, to train a more accurate and robust DETR model. This new
training scheme can easily incorporate the predicted boxes from multiple
teacher detectors, each of which provides parallel supervisions to the student
DETR. Our strategy introduces no additional parameters and adds negligible
computational cost to the original detector during training. During inference,
Teach-DETR brings zero additional overhead and maintains the merit of requiring
no non-maximum suppression. Extensive experiments show that our method leads to
consistent improvement for various DETR-based detectors. Specifically, we
improve the state-of-the-art detector DINO with Swin-Large backbone, 4 scales
of feature maps and 36-epoch training schedule, from 57.8% to 58.9% in terms of
mean average precision on MSCOCO 2017 validation set. Code will be available at
https://github.com/LeonHLJ/Teach-DETR.
