  With the advent of big data applications, which tends to have longer
execution time, choosing the right cloud VM to run these applications has
significant performance as well as economic implications. For example, in our
large-scale empirical study of 107 different workloads on three popular big
data systems, we found that a wrong choice can lead to a 20 times slowdown or
an increase in cost by 10 times.
  Bayesian optimization is a technique for optimizing expensive (black-box)
functions. Previous attempts have only used instance-level information (such as
# of cores, memory size) which is not sufficient to represent the search space.
In this work, we discover that this may lead to the fragility problem---either
incurs high search cost or finds only the sub-optimal solution. The central
insight of this paper is to use low-level performance information to augment
the process of Bayesian Optimization. Our novel low-level augmented Bayesian
Optimization is rarely worse than current practices and often performs much
better (in 46 of 107 cases). Further, it significantly reduces the search cost
in nearly half of our case studies.
  Based on this work, we conclude that it is often insufficient to use
general-purpose off-the-shelf methods for configuring cloud instances without
augmenting those methods with essential systems knowledge such as CPU
utilization, working memory size and I/O wait time.
