  Standard neural networks are often overconfident when presented with data
outside the training distribution. We introduce HyperGAN, a new generative
model for learning a distribution of neural network parameters. HyperGAN does
not require restrictive assumptions on priors, and networks sampled from it can
be used to quickly create very large and diverse ensembles. HyperGAN employs a
novel mixer to project prior samples to a latent space with correlated
dimensions, and samples from the latent space are then used to generate weights
for each layer of a deep neural network. We show that HyperGAN can learn to
generate parameters which label the MNIST and CIFAR-10 datasets with
competitive performance to fully supervised learning, while learning a rich
distribution of effective parameters. We also show that HyperGAN can also
provide better uncertainty estimates than standard ensembles by evaluating on
out of distribution data as well as adversarial examples.
