  Recently, the unmanned aerial vehicles (UAVs) have been widely used in
real-time sensing applications over cellular networks, which sense the
conditions of the tasks and transmit the real-time sensory data to the base
station (BS). The performance of a UAV is determined by the performance of both
its sensing and transmission processes, which are influenced by the trajectory
of the UAV. However, it is challenging for UAVs to design their trajectories
efficiently, since they work in a dynamic environment. To tackle this
challenge, in this paper, we adopt the reinforcement learning framework to
solve the UAV trajectory design problem in a decentralized manner. To
coordinate multiple UAVs performing the real-time sensing tasks, we first
propose a sense-and-send protocol, and analyze the probability for successful
valid data transmission using nested Markov chains. Then, we formulate the
decentralized trajectory design problem and propose an enhanced multi-UAV
Q-learning algorithm to solve this problem. Simulation results show that the
proposed enhanced multi-UAV Q-learning algorithm converges faster and achieves
higher utilities for the UAVs in the real-time task-sensing scenarios.
