  In this paper, we present a novel distributed UAVs beam reforming approach to
dynamically form and reform a space-selective beam path in addressing the
coexistence with satellite and terrestrial communications. Despite the unique
advantage to support wider coverage in UAV-enabled cellular communications, the
challenges reside in the array responses' sensitivity to random rotational
motion and the hovering nature of the UAVs. A model-free reinforcement learning
(RL) based unified UAV beam selection and tracking approach is presented to
effectively realize the dynamic distributed and collaborative beamforming. The
combined impact of the UAVs' hovering and rotational motions is considered
while addressing the impairment due to the interference from the orbiting
satellites and neighboring networks. The main objectives of this work are
two-fold: first, to acquire the channel awareness to uncover its impairments;
second, to overcome the beam distortion to meet the quality of service (QoS)
requirements. To overcome the impact of the interference and to maximize the
beamforming gain, we define and apply a new optimal UAV selection algorithm
based on the brute force criteria. Results demonstrate that the detrimental
effects of the channel fading and the interference from the orbiting satellites
and neighboring networks can be overcome using the proposed approach.
Subsequently, an RL algorithm based on Deep Q-Network (DQN) is developed for
real-time beam tracking. By augmenting the system with the impairments due to
hovering and rotational motion, we show that the proposed DQN algorithm can
reform the beam in real-time with negligible error. It is demonstrated that the
proposed DQN algorithm attains an exceptional performance improvement. We show
that it requires a few iterations only for fine-tuning its parameters without
observing any plateaus irrespective of the hovering tolerance.
