  In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted
Internet-of-Things (IoT) system in a sophisticated three-dimensional (3D)
environment, where the UAV's trajectory is optimized to efficiently collect
data from multiple IoT ground nodes. Unlike existing approaches focusing only
on a simplified two-dimensional scenario and the availability of perfect
channel state information (CSI), this paper considers a practical 3D urban
environment with imperfect CSI, where the UAV's trajectory is designed to
minimize data collection completion time subject to practical throughput and
flight movement constraints. Specifically, inspired from the state-of-the-art
deep reinforcement learning approaches, we leverage the twin-delayed deep
deterministic policy gradient (TD3) to design the UAV's trajectory and present
a TD3-based trajectory design for completion time minimization (TD3-TDCTM)
algorithm. In particular, we set an additional information, i.e., the merged
pheromone, to represent the state information of UAV and environment as a
reference of reward which facilitates the algorithm design. By taking the
service statuses of IoT nodes, the UAV's position, and the merged pheromone as
input, the proposed algorithm can continuously and adaptively learn how to
adjust the UAV's movement strategy. By interacting with the external
environment in the corresponding Markov decision process, the proposed
algorithm can achieve a near-optimal navigation strategy. Our simulation
results show the superiority of the proposed TD3-TDCTM algorithm over three
conventional non-learning based baseline methods.
