  Real-world black-box optimization often involves time-consuming or costly
experiments and simulations. Multi-fidelity optimization (MFO) stands out as a
cost-effective strategy that balances high-fidelity accuracy with computational
efficiency through a hierarchical fidelity approach. This survey presents a
systematic exploration of MFO, underpinned by a novel text mining framework
based on a pre-trained language model. We delve deep into the foundational
principles and methodologies of MFO, focusing on three core components --
multi-fidelity surrogate models, fidelity management strategies, and
optimization techniques. Additionally, this survey highlights the diverse
applications of MFO across several key domains, including machine learning,
engineering design optimization, and scientific discovery, showcasing the
adaptability and effectiveness of MFO in tackling complex computational
challenges. Furthermore, we also envision several emerging challenges and
prospects in the MFO landscape, spanning scalability, the composition of lower
fidelities, and the integration of human-in-the-loop approaches at the
algorithmic level. We also address critical issues related to benchmarking and
the advancement of open science within the MFO community. Overall, this survey
aims to catalyze further research and foster collaborations in MFO, setting the
stage for future innovations and breakthroughs in the field.
