  Bayesian optimization (BO) is an efficient framework for optimization of
black-box objectives when function evaluations are costly and gradient
information is not easily accessible. BO has been successfully applied to
automate the task of hyperparameter optimization (HPO) in machine learning (ML)
models with the primary objective of optimizing predictive performance on
held-out data. In recent years, however, with ever-growing model sizes, the
energy cost associated with model training has become an important factor for
ML applications. Here we evaluate Constrained Bayesian Optimization (CBO) with
the primary objective of minimizing energy consumption and subject to the
constraint that the generalization performance is above some threshold. We
evaluate our approach on regression and classification tasks and demonstrate
that CBO achieves lower energy consumption without compromising the predictive
performance of ML models.
