  In recent years generative models of visual data have made a great progress,
and now they are able to produce images of high quality and diversity. In this
work we study representations learnt by a GAN generator. First, we show that
these representations can be easily projected onto semantic segmentation map
using a lightweight decoder. We find that such semantic projection can be
learnt from just a few annotated images. Based on this finding, we propose
LayerMatch scheme for approximating the representation of a GAN generator that
can be used for unsupervised domain-specific pretraining. We consider the
semi-supervised learning scenario when a small amount of labeled data is
available along with a large unlabeled dataset from the same domain. We find
that the use of LayerMatch-pretrained backbone leads to superior accuracy
compared to standard supervised pretraining on ImageNet. Moreover, this simple
approach also outperforms recent semi-supervised semantic segmentation methods
that use both labeled and unlabeled data during training. Source code for
reproducing our experiments will be available at the time of publication.
