  In this paper, we develop an algorithm for joint handover and beam tracking
in millimeter-wave (mmWave) networks. The aim is to provide a reliable
connection in terms of the achieved throughput along the trajectory of the
mobile user while preventing frequent handovers. We model the association
problem as an optimization problem and propose a reinforcement learning-based
solution. Our approach learns whether and when beam tracking and handover
should be performed and chooses the target base stations. In the case of beam
tracking, we propose a tracking algorithm based on measuring a small spatial
neighbourhood of the optimal beams in the previous time slot. Simulation
results in an outdoor environment show the superior performance of our proposed
solution in achievable throughput and the number of handovers needed in
comparison to a multi-connectivity baseline and a learning-based handover
baseline.
