  Bayesian Optimization (BO) is a surrogate-assisted global optimization
technique that has been successfully applied in various fields, e.g., automated
machine learning and design optimization. Built upon a so-called
infill-criterion and Gaussian Process regression (GPR), the BO technique
suffers from a substantial computational complexity and hampered convergence
rate as the dimension of the search spaces increases. Scaling up BO for
high-dimensional optimization problems remains a challenging task. In this
paper, we propose to tackle the scalability of BO by hybridizing it with a
Principal Component Analysis (PCA), resulting in a novel PCA-assisted BO
(PCA-BO) algorithm. Specifically, the PCA procedure learns a linear
transformation from all the evaluated points during the run and selects
dimensions in the transformed space according to the variability of evaluated
points. We then construct the GPR model, and the infill-criterion in the space
spanned by the selected dimensions. We assess the performance of our PCA-BO in
terms of the empirical convergence rate and CPU time on multi-modal problems
from the COCO benchmark framework. The experimental results show that PCA-BO
can effectively reduce the CPU time incurred on high-dimensional problems, and
maintains the convergence rate on problems with an adequate global structure.
PCA-BO therefore provides a satisfactory trade-off between the convergence rate
and computational efficiency opening new ways to benefit from the strength of
BO approaches in high dimensional numerical optimization.
