  Millimeter-wave (mmWave) hybrid analog-digital beamforming is a promising
approach to satisfy the low-latency constraint in multiple unmanned aerial
vehicles (UAVs) systems, which serve as network infrastructure for flexible
deployment. However, in highly dynamic multi-UAV environments, analog beam
tracking becomes a critical challenge. The overhead of additional pilot
transmission at the price of spectral efficiency is shown necessary to achieve
high resilience in operation. An efficient method to deal with high dynamics of
UAVs applies machine learning, particularly Q-learning, to analog beam
tracking. The proposed Q-learning-based beam tracking scheme uses current/past
observations to design rewards from environments to facilitate prediction,
which significantly increases the efficiency of data transmission and beam
switching. Given the selected analog beams, the goal of digital beamforming is
to maximize the SINR. The received pilot signals are utilized to approximate
the desired signal and interference power, which yield the SINR measurements as
well as the optimal digital weights. Since the selected analog beams based on
the received power do not guarantee the hybrid beamforming achieving the
maximization SINR, we therefore reserve additional analog beams as candidates
during the beam tracking. The combination of analog beams with their digital
weights achieving the maximum SINR consequently provides the optimal solution
to the hybrid beamforming.
