  The existing solutions for object detection distillation rely on the
availability of both a teacher model and ground-truth labels. We propose a new
perspective to relax this constraint. In our framework, a student is first
trained with pseudo labels generated by the teacher, and then fine-tuned using
labeled data, if any available. Extensive experiments demonstrate improvements
over existing object detection distillation algorithms. In addition, decoupling
the teacher and ground-truth distillation in this framework provides
interesting properties such: as 1) using unlabeled data to further improve the
student's performance, 2) combining multiple teacher models of different
architectures, even with different object categories, and 3) reducing the need
for labeled data (with only 20% of COCO labels, this method achieves the same
performance as the model trained on the entire set of labels). Furthermore, a
by-product of this approach is the potential usage for domain adaptation. We
verify these properties through extensive experiments.
