  While generative adversarial networks (GANs) have revolutionized machine
learning, a number of open questions remain to fully understand them and
exploit their power. One of these questions is how to efficiently achieve
proper diversity and sampling of the multi-mode data space. To address this, we
introduce BasisGAN, a stochastic conditional multi-mode image generator. By
exploiting the observation that a convolutional filter can be well approximated
as a linear combination of a small set of basis elements, we learn a
plug-and-played basis generator to stochastically generate basis elements, with
just a few hundred of parameters, to fully embed stochasticity into
convolutional filters. By sampling basis elements instead of filters, we
dramatically reduce the cost of modeling the parameter space with no sacrifice
on either image diversity or fidelity. To illustrate this proposed
plug-and-play framework, we construct variants of BasisGAN based on
state-of-the-art conditional image generation networks, and train the networks
by simply plugging in a basis generator, without additional auxiliary
components, hyperparameters, or training objectives. The experimental success
is complemented with theoretical results indicating how the perturbations
introduced by the proposed sampling of basis elements can propagate to the
appearance of generated images.
