  Knowledge distillation has made remarkable achievements in model compression.
However, most existing methods require the original training data, which is
usually unavailable due to privacy and security issues. In this paper, we
propose a conditional generative data-free knowledge distillation (CGDD)
framework for training lightweight networks without any training data. This
method realizes efficient knowledge distillation based on conditional image
generation. Specifically, we treat the preset labels as ground truth to train a
conditional generator in a semi-supervised manner. The trained generator can
produce specified classes of training images. For training the student network,
we force it to extract the knowledge hidden in teacher feature maps, which
provide crucial cues for the learning process. Moreover, an adversarial
training framework for promoting distillation performance is constructed by
designing several loss functions. This framework helps the student model to
explore larger data space. To demonstrate the effectiveness of the proposed
method, we conduct extensive experiments on different datasets. Compared with
other data-free works, our work obtains state-of-the-art results on CIFAR100,
Caltech101, and different versions of ImageNet datasets. The codes will be
released.
