  We propose an extrinsic Bayesian optimization (eBO) framework for general
optimization problems on manifolds. Bayesian optimization algorithms build a
surrogate of the objective function by employing Gaussian processes and
quantify the uncertainty in that surrogate by deriving an acquisition function.
This acquisition function represents the probability of improvement based on
the kernel of the Gaussian process, which guides the search in the optimization
process. The critical challenge for designing Bayesian optimization algorithms
on manifolds lies in the difficulty of constructing valid covariance kernels
for Gaussian processes on general manifolds. Our approach is to employ
extrinsic Gaussian processes by first embedding the manifold onto some higher
dimensional Euclidean space via equivariant embeddings and then constructing a
valid covariance kernel on the image manifold after the embedding. This leads
to efficient and scalable algorithms for optimization over complex manifolds.
Simulation study and real data analysis are carried out to demonstrate the
utilities of our eBO framework by applying the eBO to various optimization
problems over manifolds such as the sphere, the Grassmannian, and the manifold
of positive definite matrices.
