  Unmanned aerial vehicle (UAV)-assisted data collection has been emerging as a
prominent application due to its flexibility, mobility, and low operational
cost. However, under the dynamic and uncertainty of IoT data collection and
energy replenishment processes, optimizing the performance for UAV collectors
is a very challenging task. Thus, this paper introduces a novel framework that
jointly optimizes the flying speed and energy replenishment for each UAV to
significantly improve the data collection performance. Specifically, we first
develop a Markov decision process to help the UAV automatically and dynamically
make optimal decisions under the dynamics and uncertainties of the environment.
We then propose a highly-effective reinforcement learning algorithm leveraging
deep Q-learning, double deep Q-learning, and a deep dueling neural network
architecture to quickly obtain the UAV's optimal policy. The core ideas of this
algorithm are to estimate the state values and action advantages separately and
simultaneously and to employ double estimators for estimating the action
values. Thus, these proposed techniques can stabilize the learning process and
effectively address the overestimation problem of conventional Q-learning
algorithms. To further reduce the learning time as well as significantly
improve learning quality, we develop advanced transfer learning techniques to
allow UAVs to ``share'' and ``transfer'' learning knowledge. Extensive
simulations demonstrate that our proposed solution can improve the average data
collection performance of the system up to 200% compared with those of current
methods.
