  Unmanned aerial vehicles (UAVs) are expected to be an integral part of
wireless networks, and determining collision-free trajectories for multiple
UAVs while satisfying requirements of connectivity with ground base stations
(GBSs) is a challenging task. In this paper, we first reformulate the multi-UAV
trajectory optimization problem with collision avoidance and wireless
connectivity constraints as a sequential decision making problem in the
discrete time domain. We, then, propose a decentralized deep reinforcement
learning approach to solve the problem. More specifically, a value network is
developed to encode the expected time to destination given the agent's joint
state (including the agent's information, the nearby agents' observable
information, and the locations of the nearby GBSs). A
signal-to-interference-plus-noise ratio (SINR)-prediction neural network is
also designed, using accumulated SINR measurements obtained when interacting
with the cellular network, to map the GBSs' locations into the SINR levels in
order to predict the UAV's SINR. Numerical results show that with the value
network and SINR-prediction network, real-time navigation for multi-UAVs can be
efficiently performed in various environments with high success rate.
