  We study the GAN conditioning problem, whose goal is to convert a pretrained
unconditional GAN into a conditional GAN using labeled data. We first identify
and analyze three approaches to this problem -- conditional GAN training from
scratch, fine-tuning, and input reprogramming. Our analysis reveals that when
the amount of labeled data is small, input reprogramming performs the best.
Motivated by real-world scenarios with scarce labeled data, we focus on the
input reprogramming approach and carefully analyze the existing algorithm.
After identifying a few critical issues of the previous input reprogramming
approach, we propose a new algorithm called InRep+. Our algorithm InRep+
addresses the existing issues with the novel uses of invertible neural networks
and Positive-Unlabeled (PU) learning. Via extensive experiments, we show that
InRep+ outperforms all existing methods, particularly when label information is
scarce, noisy, and/or imbalanced. For instance, for the task of conditioning a
CIFAR10 GAN with 1% labeled data, InRep+ achieves an average Intra-FID of
76.24, whereas the second-best method achieves 114.51.
