  We study decentralized multi-agent learning in bipartite queueing systems, a
standard model for service systems. In particular, N agents request service
from K servers in a fully decentralized way, i.e, by running the same algorithm
without communication. Previous decentralized algorithms are restricted to
symmetric systems, have performance that is degrading exponentially in the
number of servers, require communication through shared randomness and unique
agent identities, and are computationally demanding. In contrast, we provide a
simple learning algorithm that, when run decentrally by each agent, leads the
queueing system to have efficient performance in general asymmetric bipartite
queueing systems while also having additional robustness properties. Along the
way, we provide the first provably efficient UCB-based algorithm for the
centralized case of the problem.
