  Query processing over big data is ubiquitous in modern clouds, where the
system takes care of picking both the physical query execution plans and the
resources needed to run those plans, using a cost-based query optimizer. A good
cost model, therefore, is akin to better resource efficiency and lower
operational costs. Unfortunately, the production workloads at Microsoft show
that costs are very complex to model for big data systems. In this work, we
investigate two key questions: (i) can we learn accurate cost models for big
data systems, and (ii) can we integrate the learned models within the query
optimizer. To answer these, we make three core contributions. First, we exploit
workload patterns to learn a large number of individual cost models and combine
them to achieve high accuracy and coverage over a long period. Second, we
propose extensions to Cascades framework to pick optimal resources, i.e, number
of containers, during query planning. And third, we integrate the learned cost
models within the Cascade-style query optimizer of SCOPE at Microsoft. We
evaluate the resulting system, Cleo, in a production environment using both
production and TPC-H workloads. Our results show that the learned cost models
are 2 to 3 orders of magnitude more accurate, and 20X more correlated with the
actual runtimes, with a large majority (70%) of the plan changes leading to
substantial improvements in latency as well as resource usage.
