  Many researchers have sought ways of model compression to reduce the size of
a deep neural network (DNN) with minimal performance degradation in order to
use DNNs in embedded systems. Among the model compression methods, a method
called knowledge transfer is to train a student network with a stronger teacher
network. In this paper, we propose a novel knowledge transfer method which uses
convolutional operations to paraphrase teacher's knowledge and to translate it
for the student. This is done by two convolutional modules, which are called a
paraphraser and a translator. The paraphraser is trained in an unsupervised
manner to extract the teacher factors which are defined as paraphrased
information of the teacher network. The translator located at the student
network extracts the student factors and helps to translate the teacher factors
by mimicking them. We observed that our student network trained with the
proposed factor transfer method outperforms the ones trained with conventional
knowledge transfer methods.
