  The success of state-of-the-art deep neural networks heavily relies on the
presence of large-scale labelled datasets, which are extremely expensive and
time-consuming to annotate. This paper focuses on tackling semi-supervised part
segmentation tasks by generating high-quality images with a pre-trained GAN and
labelling the generated images with an automatic annotator. In particular, we
formulate the annotator learning as a learning-to-learn problem. Given a
pre-trained GAN, the annotator learns to label object parts in a set of
randomly generated images such that a part segmentation model trained on these
synthetic images with their predicted labels obtains low segmentation error on
a small validation set of manually labelled images. We further reduce this
nested-loop optimization problem to a simple gradient matching problem and
efficiently solve it with an iterative algorithm. We show that our method can
learn annotators from a broad range of labelled images including real images,
generated images, and even analytically rendered images. Our method is
evaluated with semi-supervised part segmentation tasks and significantly
outperforms other semi-supervised competitors when the amount of labelled
examples is extremely limited.
