  Bayesian Optimization, the application of Bayesian function approximation to
finding optima of expensive functions, has exploded in popularity in recent
years. In particular, much attention has been paid to improving its efficiency
on problems with many parameters to optimize. This attention has trickled down
to the workhorse of high dimensional BO, high dimensional Gaussian process
regression, which is also of independent interest. The great flexibility that
the Gaussian process prior implies is a boon when modeling complicated, low
dimensional surfaces but simply says too little when dimension grows too large.
A variety of structural model assumptions have been tested to tame high
dimensions, from variable selection and additive decomposition to low
dimensional embeddings and beyond. Most of these approaches in turn require
modifications of the acquisition function optimization strategy as well. Here
we review the defining structural model assumptions and discuss the benefits
and drawbacks of these approaches in practice.
