  In recent years, unmanned aerial vehicles (UAVs) have found numerous sensing
applications, which are expected to add billions of dollars to the world
economy in the next decade. To further improve the Quality-of-Service (QoS) in
such applications, the 3rd Generation Partnership Project (3GPP) has considered
the adoption of terrestrial cellular networks to support UAV sensing services,
also known as the cellular Internet of UAVs. In this paper, we consider a
cellular Internet of UAVs, where the sensory data can be transmitted either to
base station (BS) via cellular links, or to mobile devices by underlay
UAV-to-Device (U2D) communications. To evaluate the freshness of data, the age
of information (AoI) is adopted, in which a lower AoI implies fresher data.
Since UAVs' AoIs are determined by their trajectories during sensing and
transmission, we investigate the AoI minimization problem for UAVs by designing
their trajectories. This problem is a Markov decision problem (MDP) with an
infinite state-action space, and thus we utilize multi-agent deep reinforcement
learning (DRL) to approximate the state-action space. Then, we propose a
multi-UAV trajectory design algorithm to solve this problem. Simulation results
show that our algorithm achieves a lower AoI than greedy algorithm and policy
gradient algorithm.
