  We present an analysis of large-scale load balancing systems, where the
processing time distribution of tasks depends on both the task and server
types. Our study focuses on the asymptotic regime, where the number of servers
and task types tend to infinity in proportion. In heterogeneous environments,
commonly used load balancing policies such as Join Fastest Idle Queue and Join
Fastest Shortest Queue exhibit poor performance and even shrink the stability
region. Interestingly, prior to this work, finding a scalable policy with a
provable performance guarantee in this setup remained an open question.
  To address this gap, we propose and analyze two asymptotically delay-optimal
dynamic load balancing policies. The first policy efficiently reserves the
processing capacity of each server for ``good" tasks and routes tasks using the
vanilla Join Idle Queue policy. The second policy, called the speed-priority
policy, significantly increases the likelihood of assigning tasks to the
respective ``good" servers capable of processing them at high speeds. By
leveraging a framework inspired by the graphon literature and employing the
mean-field method and stochastic coupling arguments, we demonstrate that both
policies achieve asymptotic zero queuing. Specifically, as the system scales,
the probability of a typical task being assigned to an idle server approaches
1.
