  Black-box optimization problems often require simultaneously optimizing
different types of variables, such as continuous, integer, and categorical
variables. Unlike integer variables, categorical variables do not necessarily
have a meaningful order, and the discretization approach of continuous
variables does not work well. Although several Bayesian optimization methods
can deal with mixed-category black-box optimization (MC-BBO), they suffer from
a lack of scalability to high-dimensional problems and internal computational
cost. This paper proposes CatCMA, a stochastic optimization method for MC-BBO
problems, which employs the joint probability distribution of multivariate
Gaussian and categorical distributions as the search distribution. CatCMA
updates the parameters of the joint probability distribution in the natural
gradient direction. CatCMA also incorporates the acceleration techniques used
in the covariance matrix adaptation evolution strategy (CMA-ES) and the
stochastic natural gradient method, such as step-size adaptation and learning
rate adaptation. In addition, we restrict the ranges of the categorical
distribution parameters by margin to prevent premature convergence and
analytically derive a promising margin setting. Numerical experiments show that
the performance of CatCMA is superior and more robust to problem dimensions
compared to state-of-the-art Bayesian optimization algorithms.
