  In this paper, we develop a hierarchical aerial computing framework composed
of high altitude platform (HAP) and unmanned aerial vehicles (UAVs) to compute
the fully offloaded tasks of terrestrial mobile users which are connected
through an uplink non-orthogonal multiple access (UL-NOMA). To better assess
the freshness of information in computation-intensive applications the
criterion of age of information (AoI) is considered. In particular, the problem
is formulated to minimize the average AoI of users with elastic tasks, by
adjusting UAVs trajectory and resource allocation on both UAVs and HAP, which
is restricted by the channel state information (CSI) uncertainty and multiple
resource constraints of UAVs and HAP. In order to solve this non-convex
optimization problem, two methods of multi-agent deep deterministic policy
gradient (MADDPG) and federated reinforcement learning (FRL) are proposed to
design the UAVs trajectory, and obtain channel, power, and CPU allocations. It
is shown that task scheduling significantly reduces the average AoI. This
improvement is more pronounced for larger task sizes. On one hand, it is shown
that power allocation has a marginal effect on the average AoI compared to
using full transmission power for all users. Compared with traditional
transmission schemes, the simulation results show our scheduling scheme results
in a substantial improvement in average AoI.
