  Recent technological advancements in space, air, and ground components have
made possible a new network paradigm called space-air-ground integrated network
(SAGIN). Unmanned aerial vehicles (UAVs) play a key role in SAGINs. However,
due to UAVs' high dynamics and complexity, real-world deployment of a SAGIN
becomes a significant barrier to realizing such SAGINs. UAVs are expected to
meet key performance requirements with limited maneuverability and resources
with space and terrestrial components. Therefore, employing UAVs in various
usage scenarios requires well-designed planning in algorithmic approaches. This
paper provides an essential review and analysis of recent learning algorithms
in a UAV-assisted SAGIN. We consider possible reward functions and discuss the
state-of-the-art algorithms for optimizing the reward functions, including
Q-learning, deep Q-learning, multi-armed bandit, particle swarm optimization,
and satisfaction-based learning algorithms. Unlike other survey papers, we
focus on the methodological perspective of the optimization problem, applicable
to various missions on a SAGIN. We consider real-world configurations and the
2-dimensional (2D) and 3-dimensional (3D) UAV trajectories to reflect
deployment cases. Our simulations suggest the 3D satisfaction-based learning
algorithm outperforms other approaches in most cases. With open challenges
discussed at the end, we aim to provide design and deployment guidelines for
UAV-assisted SAGINs.
