  Quadcopters have been studied for decades thanks to their maneuverability and
capability of operating in a variety of circumstances. However, quadcopters
suffer from dynamical nonlinearity, actuator saturation, as well as sensor
noise that make it challenging and time consuming to obtain accurate dynamic
models and achieve satisfactory control performance. Fortunately, deep
reinforcement learning came and has shown significant potential in system
modelling and control of autonomous multirotor aerial vehicles, with recent
advancements in deployment, performance enhancement, and generalization. In
this paper, an end-to-end deep reinforcement learning-based controller for
quadcopters is proposed that is secure for real-world implementation,
data-efficient, and free of human gain adjustments. First, a novel
actor-critic-based architecture is designed to map the robot states directly to
the motor outputs. Then, a quadcopter dynamics-based simulator was devised to
facilitate the training of the controller policy. Finally, the trained policy
is deployed on a real Crazyflie nano quadrotor platform, without any additional
fine-tuning process. Experimental results show that the quadcopter exhibits
satisfactory performance as it tracks a given complicated trajectory, which
demonstrates the effectiveness and feasibility of the proposed method and
signifies its capability in filling the simulation-to-reality gap.
