  In this paper we investigate the convergence properties of a variant of the
Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Our study is based on
the recent theoretical foundation that the pure rank-mu update CMA-ES performs
the natural gradient descent on the parameter space of Gaussian distributions.
We derive a novel variant of the natural gradient method where the parameters
of the Gaussian distribution are updated along the natural gradient to improve
a newly defined function on the parameter space. We study this algorithm on
composites of a monotone function with a convex quadratic function. We prove
that our algorithm adapts the covariance matrix so that it becomes proportional
to the inverse of the Hessian of the original objective function. We also show
the speed of covariance matrix adaptation and the speed of convergence of the
parameters. We introduce a stochastic algorithm that approximates the natural
gradient with finite samples and present some simulated results to evaluate how
precisely the stochastic algorithm approximates the deterministic, ideal one
under finite samples and to see how similarly our algorithm and the CMA-ES
perform.
