  In this paper, a deep reinforcement learning (DRL) method is proposed to
address the problem of UAV navigation in an unknown environment. However, DRL
algorithms are limited by the data efficiency problem as they typically require
a huge amount of data before they reach a reasonable performance. To speed up
the DRL training process, we developed a novel learning framework which
combines imitation learning and reinforcement learning and building upon Twin
Delayed DDPG (TD3) algorithm. We newly introduced both policy and Q-value
network are learned using the expert demonstration during the imitation phase.
To tackle the distribution mismatch problem transfer from imitation to
reinforcement learning, both TD-error and decayed imitation loss are used to
update the pre-trained network when start interacting with the environment. The
performances of the proposed algorithm are demonstrated on the challenging 3D
UAV navigation problem using depth cameras and sketched in a variety of
simulation environments.
