  Cellular-connected unmanned aerial vehicle (UAV) swarm is a promising
solution for diverse applications, including cargo delivery and traffic
control. However, it is still challenging to communicate with and control the
UAV swarm with high reliability, low latency, and high energy efficiency. In
this paper, we propose a two-phase command and control (C&C) transmission
scheme in a cellular-connected UAV swarm network, where the ground base station
(GBS) broadcasts the common C&C message in Phase I. In Phase II, the UAVs that
have successfully decoded the C&C message will relay the message to the rest of
UAVs via device-to-device (D2D) communications in either broadcast or unicast
mode, under latency and energy constraints. To maximize the number of UAVs that
receive the message successfully within the latency and energy constraints, we
formulate the problem as a Constrained Markov Decision Process to find the
optimal policy. To address this problem, we propose a decentralized constrained
graph attention multi-agent Deep-Q-network (DCGA-MADQN) algorithm based on
Lagrangian primal-dual policy optimization, where a PID-controller algorithm is
utilized to update the Lagrange Multiplier. Simulation results show that our
algorithm could maximize the number of UAVs that successfully receive the
common C&C under energy constraints.
