  The space-air-ground integrated network (SAGIN) has become a crucial research
direction in future wireless communications due to its ubiquitous coverage,
rapid and flexible deployment, and multi-layer cooperation capabilities.
However, integrating hierarchical federated learning (HFL) with edge computing
and SAGINs remains a complex open issue to be resolved. This paper proposes a
novel framework for applying HFL in SAGINs, utilizing aerial platforms and low
Earth orbit (LEO) satellites as edge servers and cloud servers, respectively,
to provide multi-layer aggregation capabilities for HFL. The proposed system
also considers the presence of inter-satellite links (ISLs), enabling
satellites to exchange federated learning models with each other. Furthermore,
we consider multiple different computational tasks that need to be completed
within a limited satellite service time. To maximize the convergence
performance of all tasks while ensuring fairness, we propose the use of the
distributional soft-actor-critic (DSAC) algorithm to optimize resource
allocation in the SAGIN and aggregation weights in HFL. Moreover, we address
the efficiency issue of hybrid action spaces in deep reinforcement learning
(DRL) through a decoupling and recoupling approach, and design a new dynamic
adjusting reward function to ensure fairness among multiple tasks in federated
learning. Simulation results demonstrate the superiority of our proposed
algorithm, consistently outperforming baseline approaches and offering a
promising solution for addressing highly complex optimization problems in
SAGINs.
