  This paper studies the trajectory optimization problem for an aerial vehicle
with the mission of flying between a pair of given initial and final locations.
The objective is to minimize the travel time of the aerial vehicle ensuring
that the communication connectivity constraint required for the safe operation
of the aerial vehicle is satisfied. We consider two different criteria for the
connectivity constraint of the aerial vehicle which leads to two different
scenarios. In the first scenario, we assume that the maximum continuous time
duration that the aerial vehicle is out of the coverage of the ground base
stations (GBSs) is limited to a given threshold. In the second scenario,
however, we assume that the total time periods that the aerial vehicle is not
covered by the GBSs is restricted. Based on these two constraints, we formulate
two trajectory optimization problems. To solve these non-convex problems, we
use an approach based on the double Q-learning method which is a model-free
reinforcement learning technique and unlike the existing algorithms does not
need perfect knowledge of the environment. Moreover, in contrast to the
well-known Q-learning technique, our double Q-learning algorithm does not
suffer from the over-estimation issue. Simulation results show that although
our algorithm does not require prior information of the environment, it works
well and shows near optimal performance.
