  Knowledge distillation (KD) enhances the performance of a student network by
allowing it to learn the knowledge transferred from a teacher network
incrementally. Existing methods dynamically adjust the temperature to enable
the student network to adapt to the varying learning difficulties at different
learning stages of KD. KD is a continuous process, but when adjusting the
temperature, these methods consider only the immediate benefits of the
operation in the current learning phase and fail to take into account its
future returns. To address this issue, we formulate the adjustment of
temperature as a sequential decision-making task and propose a method based on
reinforcement learning, termed RLKD. Importantly, we design a novel state
representation to enable the agent to make more informed action (i.e. instance
temperature adjustment). To handle the problem of delayed rewards in our method
due to the KD setting, we explore an instance reward calibration approach. In
addition,we devise an efficient exploration strategy that enables the agent to
learn valuable instance temperature adjustment policy more efficiently. Our
framework can serve as a plug-and-play technique to be inserted into various KD
methods easily, and we validate its effectiveness on both image classification
and object detection tasks. Our project is at
https://www.zayx.me/ITKD.github.io/.
