  We study the problem of using low computational cost to automate the choices
of learners and hyperparameters for an ad-hoc training dataset and error
metric, by conducting trials of different configurations on the given training
data. We investigate the joint impact of multiple factors on both trial cost
and model error, and propose several design guidelines. Following them, we
build a fast and lightweight library FLAML which optimizes for low
computational resource in finding accurate models. FLAML integrates several
simple but effective search strategies into an adaptive system. It
significantly outperforms top-ranked AutoML libraries on a large open source
AutoML benchmark under equal, or sometimes orders of magnitude smaller budget
constraints.
