  Data-free knowledge distillation (DFKD) is a widely-used strategy for
Knowledge Distillation (KD) whose training data is not available. It trains a
lightweight student model with the aid of a large pretrained teacher model
without any access to training data. However, existing DFKD methods suffer from
inadequate and unstable training process, as they do not adjust the generation
target dynamically based on the status of the student model during learning. To
address this limitation, we propose a novel DFKD method called CuDFKD. It
teaches students by a dynamic strategy that gradually generates easy-to-hard
pseudo samples, mirroring how humans learn. Besides, CuDFKD adapts the
generation target dynamically according to the status of student model.
Moreover, We provide a theoretical analysis of the majorization minimization
(MM) algorithm and explain the convergence of CuDFKD. To measure the robustness
and fidelity of DFKD methods, we propose two more metrics, and experiments
shows CuDFKD has comparable performance to state-of-the-art (SOTA) DFKD methods
on all datasets. Experiments also present that our CuDFKD has the fastest
convergence and best robustness over other SOTA DFKD methods.
