  The deep learning revolution has greatly been accelerated by the 'hardware
lottery': Recent advances in modern hardware accelerators and compilers paved
the way for large-scale batch gradient optimization. Evolutionary optimization,
on the other hand, has mainly relied on CPU-parallelism, e.g. using Dask
scheduling and distributed multi-host infrastructure. Here we argue that also
modern evolutionary computation can significantly benefit from the massive
computational throughput provided by GPUs and TPUs. In order to better harness
these resources and to enable the next generation of black-box optimization
algorithms, we release evosax: A JAX-based library of evolution strategies
which allows researchers to leverage powerful function transformations such as
just-in-time compilation, automatic vectorization and hardware parallelization.
evosax implements 30 evolutionary optimization algorithms including
finite-difference-based, estimation-of-distribution evolution strategies and
various genetic algorithms. Every single algorithm can directly be executed on
hardware accelerators and automatically vectorized or parallelized across
devices using a single line of code. It is designed in a modular fashion and
allows for flexible usage via a simple ask-evaluate-tell API. We thereby hope
to facilitate a new wave of scalable evolutionary optimization algorithms.
