  In the current unmanned aircraft systems (UASs) for sensing services,
unmanned aerial vehicles (UAVs) transmit their sensory data to terrestrial
mobile devices over the unlicensed spectrum. However, the interference from
surrounding terminals is uncontrollable due to the opportunistic channel
access. In this paper, we consider a cellular Internet of UAVs to guarantee the
Quality-of-Service (QoS), where the sensory data can be transmitted to the
mobile devices either by UAV-to-Device (U2D) communications over cellular
networks, or directly through the base station (BS). Since UAVs' sensing and
transmission may influence their trajectories, we study the trajectory design
problem for UAVs in consideration of their sensing and transmission. This is a
Markov decision problem (MDP) with a large state-action space, and thus, we
utilize multi-agent deep reinforcement learning (DRL) to approximate the
state-action space, and then propose a multi-UAV trajectory design algorithm to
solve this problem. Simulation results show that our proposed algorithm can
achieve a higher total utility than policy gradient algorithm and single-agent
algorithm.
