  A novel framework is proposed for the trajectory design of multiple unmanned
aerial vehicles (UAVs) based on the prediction of users' mobility information.
The problem of joint trajectory design and power control is formulated for
maximizing the instantaneous sum transmit rate while satisfying the rate
requirement of users. In an effort to solve this pertinent problem, a
three-step approach is proposed which is based on machine learning techniques
to obtain both the position information of users and the trajectory design of
UAVs. Firstly, a multi-agent Q-learning based placement algorithm is proposed
for determining the optimal positions of the UAVs based on the initial location
of the users. Secondly, in an effort to determine the mobility information of
users based on a real dataset, their position data is collected from Twitter to
describe the anonymous user-trajectories in the physical world. In the
meantime, an echo state network (ESN) based prediction algorithm is proposed
for predicting the future positions of users based on the real dataset.
Thirdly, a multi-agent Q-learning based algorithm is conceived for predicting
the position of UAVs in each time slot based on the movement of users. In this
algorithm, multiple UAVs act as agents to find optimal actions by interacting
with their environment and learn from their mistakes. Additionally, we also
prove that the proposed multi-agent Q-learning based trajectory design and
power control algorithm can converge under mild conditions. Numerical results
are provided to demonstrate that as the size of the reservoir increases, the
proposed ESN approach improves the prediction accuracy. Finally, we demonstrate
that throughput gains of about 17% are achieved.
