  Many real-world optimization scenarios involve expensive evaluation with
unknown and heterogeneous costs. Cost-aware Bayesian optimization stands out as
a prominent solution in addressing these challenges. To approach the global
optimum within a limited budget in a cost-efficient manner, the design of
cost-aware acquisition functions (AFs) becomes a crucial step. However,
traditional manual design paradigm typically requires extensive domain
knowledge and involves a labor-intensive trial-and-error process. This paper
introduces EvolCAF, a novel framework that integrates large language models
(LLMs) with evolutionary computation (EC) to automatically design cost-aware
AFs. Leveraging the crossover and mutation in the algorithmic space, EvolCAF
offers a novel design paradigm, significantly reduces the reliance on domain
expertise and model training. The designed cost-aware AF maximizes the
utilization of available information from historical data, surrogate models and
budget details. It introduces novel ideas not previously explored in the
existing literature on acquisition function design, allowing for clear
interpretations to provide insights into its behavior and decision-making
process. In comparison to the well-known EIpu and EI-cool methods designed by
human experts, our approach showcases remarkable efficiency and generalization
across various tasks, including 12 synthetic problems and 3 real-world
hyperparameter tuning test sets.
