  Evolution Strategies (ES) are stochastic derivative-free optimization
algorithms whose most prominent representative, the CMA-ES algorithm, is widely
used to solve difficult numerical optimization problems. We provide the first
rigorous investigation of the linear convergence of step-size adaptive ES
involving a population and recombination, two ingredients crucially important
in practice to be robust to local irregularities or multimodality.We
investigate convergence of step-size adaptive ES with weighted recombination on
composites of strictly increasing functions with continuously differentiable
scaling-invariant functions with a global optimum. This function class includes
functions with non-convex sublevel sets and discontinuous functions. We prove
the existence of a constant r such that the logarithm of the distance to the
optimum divided by the number of iterations converges to r. The constant is
given as an expectation with respect to the stationary distribution of a Markov
chain-its sign allows to infer linear convergence or divergence of the ES and
is found numerically.Our main condition for convergence is the increase of the
expected log step-size on linear functions. In contrast to previous results,
our condition is equivalent to the almost sure geometric divergence of the
step-size on linear functions.
