  Data center schedulers operate at unprecedented scales today to accommodate
the growing demand for computing and storage power. The challenge that
schedulers face is meeting the requirements of scheduling speeds despite the
scale. To do so, most scheduler architectures use parallelism. However, these
architectures consist of multiple parallel scheduling entities that can only
utilize partial knowledge of the data center's state, as maintaining consistent
global knowledge or state would involve considerable communication overhead.
The disadvantage of scheduling without global knowledge is sub-optimal
placements-tasks may be made to wait in queues even though there are resources
available in zones outside the scope of the scheduling entity's state. This
leads to unnecessary queuing overheads and lower resource utilization of the
data center. In this paper, extend our previous work on Megha, a federated
decentralized data center scheduling architecture that uses eventual
consistency. The architecture utilizes both parallelism and an
eventually-consistent global state in each of its scheduling entities to make
fast decisions in a scalable manner. In our work, we compare Megha with 3
scheduling architectures: Sparrow, Eagle, and Pigeon, using simulation. We also
evaluate Megha's prototype on a 123-node cluster and compare its performance
with Pigeon's prototype using cluster traces. The results of our experiments
show that Megha consistently reduces delays in job completion time when
compared to other architectures.
