  Today's clusters often have to divide resources among a diverse set of jobs.
These jobs are heterogeneous both in execution time and in their rate of
arrival. Execution time heterogeneity has lead to the development of hybrid
schedulers that can schedule both short and long jobs to ensure good task
placement. However, arrival rate heterogeneity, or burstiness, remains a
problem in existing schedulers. These hybrid schedulers manage resources on
statically provisioned cluster, which can quickly be overwhelmed by bursts in
the number of arriving jobs.
  In this paper we propose CloudCoaster, a hybrid scheduler that dynamically
resizes the cluster by leveraging cheap transient servers. CloudCoaster
schedules jobs in an intelligent way that increases job performance while
reducing overall resource cost. We evaluate the effectiveness of CloudCoaster
through simulations on real-world traces and compare it against a state-of-art
hybrid scheduler. CloudCoaster improves the average queueing delay time of
short jobs by 4.8X while maintaining long job performance. In addition,
CloudCoaster reduces the short partition budget by over 29.5%.
