  In this paper, we investigate the problem of beam alignment in millimeter
wave (mmWave) systems, and design an optimal algorithm to reduce the overhead.
Specifically, due to directional communications, the transmitter and receiver
beams need to be aligned, which incurs high delay overhead since without a
priori knowledge of the transmitter/receiver location, the search space spans
the entire angular domain. This is further exacerbated under dynamic conditions
(e.g., moving vehicles) where the access to the base station (access point) is
highly dynamic with intermittent on-off periods, requiring more frequent beam
alignment and signal training. To mitigate this issue, we consider an online
stochastic optimization formulation where the goal is to maximize the
directivity gain (i.e., received energy) of the beam alignment policy within a
time period. We exploit the inherent correlation and unimodality properties of
the model, and demonstrate that contextual information improves the
performance. To this end, we propose an equivalent structured Multi-Armed
Bandit model to optimally exploit the exploration-exploitation tradeoff. In
contrast to the classical MAB models, the contextual information makes the
lower bound on regret (i.e., performance loss compared with an oracle policy)
independent of the number of beams. This is a crucial property since the number
of all combinations of beam patterns can be large in transceiver antenna
arrays, especially in massive MIMO systems. We further provide an
asymptotically optimal beam alignment algorithm, and investigate its
performance via simulations.
