  Recently, transformation-based self-supervised learning has been applied to
generative adversarial networks (GANs) to mitigate catastrophic forgetting in
the discriminator by introducing a stationary learning environment. However,
the separate self-supervised tasks in existing self-supervised GANs cause a
goal inconsistent with generative modeling due to the fact that their
self-supervised classifiers are agnostic to the generator distribution. To
address this problem, we propose a novel self-supervised GAN that unifies the
GAN task with the self-supervised task by augmenting the GAN labels (real or
fake) via self-supervision of data transformation. Specifically, the original
discriminator and self-supervised classifier are unified into a label-augmented
discriminator that predicts the augmented labels to be aware of both the
generator distribution and the data distribution under every transformation,
and then provide the discrepancy between them to optimize the generator.
Theoretically, we prove that the optimal generator could converge to replicate
the real data distribution. Empirically, we show that the proposed method
significantly outperforms previous self-supervised and data augmentation GANs
on both generative modeling and representation learning across benchmark
datasets.
