  We consider a wireless network scenario applicable to metropolitan areas with
developed public transport networks and high commute demands, where the mobile
user equipments (UEs) move along fixed and predetermined trajectories and
request to associate with millimeter-wave (mmWave) base stations (BSs). An
effective and efficient algorithm, called the Sequence Q-learning Algorithm
(SQA), is proposed to maximize the long-run average transmission rate of the
network, which is an NP-hard problem. Furthermore, the SQA tackles the
complexity issue by only allowing possible re-associations (handover of a UE
from one BS to another) at a discrete set of decision epochs and has polynomial
time complexity. This feature of the SQA also restricts too frequent handovers,
which are considered highly undesirable in mmWave networks. Moreover, we
demonstrate by extensive numerical results that the SQA can significantly
outperform the benchmark algorithms proposed in existing research by taking all
UEs' future trajectories and possible decisions into account at every decision
epoch.
