  Recently, significant progress has been made in learned image and video
compression. In particular the usage of Generative Adversarial Networks has
lead to impressive results in the low bit rate regime. However, the model size
remains an important issue in current state-of-the-art proposals and existing
solutions require significant computation effort on the decoding side. This
limits their usage in realistic scenarios and the extension to video
compression. In this paper, we demonstrate how to leverage knowledge
distillation to obtain equally capable image decoders at a fraction of the
original number of parameters. We investigate several aspects of our solution
including sequence specialization with side information for image coding.
Finally, we also show how to transfer the obtained benefits into the setting of
video compression. Overall, this allows us to reduce the model size by a factor
of 20 and to achieve 50% reduction in decoding time.
