  We study the problem of user association, i.e., determining which base
station (BS) a user should associate with, in a dense millimeter wave (mmWave)
network. In our system model, in each time slot, a user arrives with some
probability in a region with a relatively small geographical area served by a
dense mmWave network. Our goal is to devise an association policy under which,
in each time slot in which a user arrives, it is assigned to exactly one BS so
as to minimize the weighted average amount of time that users spend in the
system. The above problem is a restless multi-armed bandit problem and is
provably hard to solve. We prove that the problem is Whittle indexable, and
based on this result, propose an association policy under which an arriving
user is associated with the BS having the smallest Whittle index. Using
simulations, we show that our proposed policy outperforms several user
association policies proposed in prior work.
