  Recent progress in image-to-image translation has witnessed the success of
generative adversarial networks (GANs). However, GANs usually contain a huge
number of parameters, which lead to intolerant memory and computation
consumption and limit their deployment on edge devices. To address this issue,
knowledge distillation is proposed to transfer the knowledge from a cumbersome
teacher model to an efficient student model. However, most previous knowledge
distillation methods are designed for image classification and lead to limited
performance in image-to-image translation. In this paper, we propose
Region-aware Knowledge Distillation ReKo to compress image-to-image translation
models. Firstly, ReKo adaptively finds the crucial regions in the images with
an attention module. Then, patch-wise contrastive learning is adopted to
maximize the mutual information between students and teachers in these crucial
regions. Experiments with eight comparison methods on nine datasets demonstrate
the substantial effectiveness of ReKo on both paired and unpaired
image-to-image translation. For instance, our 7.08X compressed and 6.80X
accelerated CycleGAN student outperforms its teacher by 1.33 and 1.04 FID
scores on Horse to Zebra and Zebra to Horse, respectively. Codes will be
released on GitHub.
