  Current-day data centers and high-volume cloud services employ a broad set of
heterogeneous servers. In such settings, client requests typically arrive at
multiple entry points, and dispatching them to servers is an urgent distributed
systems problem. This paper presents an efficient solution to the load
balancing problem in such systems that improves on and overcomes problems of
previous solutions. The load balancing problem is formulated as a stochastic
optimization problem, and an efficient algorithmic solution is obtained based
on a subtle mathematical analysis of the problem. Finally, extensive evaluation
of the solution on simulated data shows that it outperforms previous solutions.
Moreover, the resulting dispatching policy can be computed very efficiently,
making the solution practically viable.
