  Remarkable achievements have been attained with Generative Adversarial
Networks (GANs) in image-to-image translation. However, due to a tremendous
amount of parameters, state-of-the-art GANs usually suffer from low efficiency
and bulky memory usage. To tackle this challenge, firstly, this paper
investigates GANs performance from a frequency perspective. The results show
that GANs, especially small GANs lack the ability to generate high-quality high
frequency information. To address this problem, we propose a novel knowledge
distillation method referred to as wavelet knowledge distillation. Instead of
directly distilling the generated images of teachers, wavelet knowledge
distillation first decomposes the images into different frequency bands with
discrete wavelet transformation and then only distills the high frequency
bands. As a result, the student GAN can pay more attention to its learning on
high frequency bands. Experiments demonstrate that our method leads to 7.08
times compression and 6.80 times acceleration on CycleGAN with almost no
performance drop. Additionally, we have studied the relation between
discriminators and generators which shows that the compression of
discriminators can promote the performance of compressed generators.
