  Training effective Generative Adversarial Networks (GANs) requires large
amounts of training data, without which the trained models are usually
sub-optimal with discriminator over-fitting. Several prior studies address this
issue by expanding the distribution of the limited training data via massive
and hand-crafted data augmentation. We handle data-limited image generation
from a very different perspective. Specifically, we design GenCo, a Generative
Co-training network that mitigates the discriminator over-fitting issue by
introducing multiple complementary discriminators that provide diverse
supervision from multiple distinctive views in training. We instantiate the
idea of GenCo in two ways. The first way is Weight-Discrepancy Co-training
(WeCo) which co-trains multiple distinctive discriminators by diversifying
their parameters. The second way is Data-Discrepancy Co-training (DaCo) which
achieves co-training by feeding discriminators with different views of the
input images (e.g., different frequency components of the input images).
Extensive experiments over multiple benchmarks show that GenCo achieves
superior generation with limited training data. In addition, GenCo also
complements the augmentation approach with consistent and clear performance
gains when combined.
