  Resided at the intersection of multi-fidelity optimization (MFO) and Bayesian
optimization (BO), MF BO has found a niche in solving expensive engineering
design optimization problems, thanks to its advantages in incorporating
physical and mathematical understandings of the problems, saving resources,
addressing exploitation-exploration trade-off, considering uncertainty, and
processing parallel computing. The increasing number of works dedicated to MF
BO suggests the need for a comprehensive review of this advanced optimization
technique. In this paper, we survey recent developments of two essential
ingredients of MF BO: Gaussian process (GP) based MF surrogates and acquisition
functions. We first categorize the existing MF modeling methods and MFO
strategies to locate MF BO in a large family of surrogate-based optimization
and MFO algorithms. We then exploit the common properties shared between the
methods from each ingredient of MF BO to describe important GP-based MF
surrogate models and review various acquisition functions. By doing so, we
expect to provide a structured understanding of MF BO. Finally, we attempt to
reveal important aspects that require further research for applications of MF
BO in solving intricate yet important design optimization problems, including
constrained optimization, high-dimensional optimization, optimization under
uncertainty, and multi-objective optimization.
