  Millimeter-wave (mmWave) communication is considered as a key enabler of
ultra-high data rates in the future cellular and wireless networks. The need
for directional communication between base stations (BSs) and users in mmWave
systems, that is achieved through beamforming, increases the complexity of the
channel estimation. Moreover, in order to provide better coverage, dense
deployment of BSs is required which causes frequent handovers and increased
association overhead. In this paper, we present an approach that jointly
addresses the beamforming and handover problems. Our solution entails an
efficient beamforming method with a minimum number of pilots and a
learning-based handover method supporting mobile scenarios. We use
reinforcement learning algorithm to learn the optimal choices of the backup BSs
in different locations of a mobile user. We show that our method provides high
rate and reliability in all locations of the user's trajectory with a minimal
number of handovers. Simulation results in an outdoor environment based on
geometric mmWave channel modeling and real building map data show the superior
performance of our proposed solution in achievable instantaneous rate and
trajectory rate.
