  In the natural world, life has found innumerable ways to survive and often
thrive. Between and even within species, each individual is in some manner
unique, and this diversity lends adaptability and robustness to life. In this
work, we aim to learn a space of diverse and high-reward policies on any given
environment. To this end, we introduce a generative model of policies, which
maps a low-dimensional latent space to an agent policy space. Our method
enables learning an entire population of agent policies, without requiring the
use of separate policy parameters. Just as real world populations can adapt and
evolve via natural selection, our method is able to adapt to changes in our
environment solely by selecting for policies in latent space. We test our
generative model's capabilities in a variety of environments, including an
open-ended grid-world and a two-player soccer environment. Code,
visualizations, and additional experiments can be found at
https://kennyderek.github.io/adap/.
