  Providing reliable connectivity to cellular-connected UAV can be very
challenging; their performance highly depends on the nature of the surrounding
environment, such as density and heights of the ground BSs. On the other hand,
tall buildings might block undesired interference signals from ground BSs,
thereby improving the connectivity between the UAVs and their serving BSs. To
address the connectivity of UAVs in such environments, this paper proposes a RL
algorithm to dynamically optimise the height of a UAV as it moves through the
environment, with the goal of increasing the throughput or spectrum efficiency
that it experiences. The proposed solution is evaluated in two settings: using
a series of generated environments where we vary the number of BS and building
densities, and in a scenario using real-world data obtained from an experiment
in Dublin, Ireland. Results show that our proposed RL-based solution improves
UAVs QoS by 6% to 41%, depending on the scenario. We also conclude that, when
flying at heights higher than the buildings, building density variation has no
impact on UAV QoS. On the other hand, BS density can negatively impact UAV QoS,
with higher numbers of BSs generating more interference and deteriorating UAV
performance.
