  Analyzing large datasets with distributed dataflow systems requires the use
of clusters. Public cloud providers offer a large variety and quantity of
resources that can be used for such clusters. However, picking the appropriate
resources in both type and number can often be challenging, as the selected
configuration needs to match a distributed dataflow job's resource demands and
access patterns. A good cluster configuration avoids hardware bottlenecks and
maximizes resource utilization, avoiding costly overprovisioning.
  We propose a collaborative approach for finding optimal cluster
configurations based on sharing and learning from historical runtime data of
distributed dataflow jobs. Collaboratively shared data can be utilized to
predict runtimes of future job executions through the use of specialized
regression models. However, training prediction models on historical runtime
data that were produced by different users and in diverse contexts requires the
models to take these contexts into account.
