  The encoding of solutions in black-box optimization is a delicate,
handcrafted balance between expressiveness and domain knowledge -- between
exploring a wide variety of solutions, and ensuring that those solutions are
useful. Our main insight is that this process can be automated by generating a
dataset of high-performing solutions with a quality diversity algorithm (here,
MAP-Elites), then learning a representation with a generative model (here, a
Variational Autoencoder) from that dataset. Our second insight is that this
representation can be used to scale quality diversity optimization to higher
dimensions -- but only if we carefully mix solutions generated with the learned
representation and those generated with traditional variation operators. We
demonstrate these capabilities by learning an low-dimensional encoding for the
inverse kinematics of a thousand joint planar arm. The results show that
learned representations make it possible to solve high-dimensional problems
with orders of magnitude fewer evaluations than the standard MAP-Elites, and
that, once solved, the produced encoding can be used for rapid optimization of
novel, but similar, tasks. The presented techniques not only scale up quality
diversity algorithms to high dimensions, but show that black-box optimization
encodings can be automatically learned, rather than hand designed.
