  Modern data science relies on data analytic pipelines to organize
interdependent computational steps. Such analytic pipelines often involve
different algorithms across multiple steps, each with its own hyperparameters.
To achieve the best performance, it is often critical to select optimal
algorithms and to set appropriate hyperparameters, which requires large
computational efforts. Bayesian optimization provides a principled way for
searching optimal hyperparameters for a single algorithm. However, many
challenges remain in solving pipeline optimization problems with
high-dimensional and highly conditional search space. In this work, we propose
Fast LineAr SearcH (FLASH), an efficient method for tuning analytic pipelines.
FLASH is a two-layer Bayesian optimization framework, which firstly uses a
parametric model to select promising algorithms, then computes a nonparametric
model to fine-tune hyperparameters of the promising algorithms. FLASH also
includes an effective caching algorithm which can further accelerate the search
process. Extensive experiments on a number of benchmark datasets have
demonstrated that FLASH significantly outperforms previous state-of-the-art
methods in both search speed and accuracy. Using 50% of the time budget, FLASH
achieves up to 20% improvement on test error rate compared to the baselines.
FLASH also yields state-of-the-art performance on a real-world application for
healthcare predictive modeling.
