  The Combined Algorithm Selection and Hyperparameters optimization (CASH)
problem is one of the fundamental problems in Automated Machine Learning
(AutoML). Motivated by the success of ensemble learning, recent AutoML systems
build post-hoc ensembles to output the final predictions instead of using the
best single learner. However, while most CASH methods focus on searching for a
single learner with the best performance, they neglect the diversity among base
learners (i.e., they may suggest similar configurations to previously evaluated
ones), which is also a crucial consideration when building an ensemble. To
tackle this issue and further enhance the ensemble performance, we propose
DivBO, a diversity-aware framework to inject explicit search of diversity into
the CASH problems. In the framework, we propose to use a diversity surrogate to
predict the pair-wise diversity of two unseen configurations. Furthermore, we
introduce a temporary pool and a weighted acquisition function to guide the
search of both performance and diversity based on Bayesian optimization.
Empirical results on 15 public datasets show that DivBO achieves the best
average ranks (1.82 and 1.73) on both validation and test errors among 10
compared methods, including post-hoc designs in recent AutoML systems and
state-of-the-art baselines for ensemble learning on CASH problems.
