  Evolutionary Strategies (ES) are a popular family of black-box zeroth-order
optimization algorithms which rely on search distributions to efficiently
optimize a large variety of objective functions. This paper investigates the
potential benefits of using highly flexible search distributions in classical
ES algorithms, in contrast to standard ones (typically Gaussians). We model
such distributions with Generative Neural Networks (GNNs) and introduce a new
training algorithm that leverages their expressiveness to accelerate the ES
procedure. We show that this tailored algorithm can readily incorporate
existing ES algorithms, and outperforms the state-of-the-art on diverse
objective functions.
