  Unmanned aerial vehicles (UAVs) are envisioned to complement the 5G
communication infrastructure in future smart cities. Hot spots easily appear in
road intersections, where effective communication among vehicles is
challenging. UAVs may serve as relays with the advantages of low price, easy
deployment, line-of-sight links, and flexible mobility. In this paper, we study
a UAV-assisted vehicular network where the UAV jointly adjusts its transmission
control (power and channel) and 3D flight to maximize the total throughput.
First, we formulate a Markov decision process (MDP) problem by modeling the
mobility of the UAV/vehicles and the state transitions. Secondly, we solve the
target problem using a deep reinforcement learning method, namely, the deep
deterministic policy gradient (DDPG), and propose three solutions with
different control objectives. Deep reinforcement learning methods obtain the
optimal policy through the interactions with the environment without knowing
the environment variables. Considering that environment variables in our
problem are unknown and unmeasurable, we choose a deep reinforcement learning
method to solve it. Moreover, considering the energy consumption of 3D flight,
we extend the proposed solutions to maximize the total throughput per unit
energy. To encourage or discourage the UAV's mobility according to its
prediction, the DDPG framework is modified, where the UAV adjusts its learning
rate automatically. Thirdly, in a simplified model with small state space and
action space, we verify the optimality of proposed algorithms. Comparing with
two baseline schemes, we demonstrate the effectiveness of proposed algorithms
in a realistic model.
