  Genetic algorithms constitute a family of black-box optimization algorithms,
which take inspiration from the principles of biological evolution. While they
provide a general-purpose tool for optimization, their particular
instantiations can be heuristic and motivated by loose biological intuition. In
this work we explore a fundamentally different approach: Given a sufficiently
flexible parametrization of the genetic operators, we discover entirely new
genetic algorithms in a data-driven fashion. More specifically, we parametrize
selection and mutation rate adaptation as cross- and self-attention modules and
use Meta-Black-Box-Optimization to evolve their parameters on a set of diverse
optimization tasks. The resulting Learned Genetic Algorithm outperforms
state-of-the-art adaptive baseline genetic algorithms and generalizes far
beyond its meta-training settings. The learned algorithm can be applied to
previously unseen optimization problems, search dimensions & evaluation
budgets. We conduct extensive analysis of the discovered operators and provide
ablation experiments, which highlight the benefits of flexible module
parametrization and the ability to transfer (`plug-in') the learned operators
to conventional genetic algorithms.
