  This paper aims to examine the potential of using the emerging deep
reinforcement learning techniques in flight control. Instead of learning from
scratch, we suggest to leverage domain knowledge available in learning to
improve learning efficiency and generalisability. More specifically, the
proposed approach fixes the autopilot structure as typical three-loop autopilot
and deep reinforcement learning is utilised to learn the autopilot gains. To
solve the flight control problem, we then formulate a Markovian decision
process with a proper reward function that enable the application of
reinforcement learning theory. Another type of domain knowledge is exploited
for defining the reward function, by shaping reference inputs in consideration
of important control objectives and using the shaped reference inputs in the
reward function. The state-of-the-art deep deterministic policy gradient
algorithm is utilised to learn an action policy that maps the observed states
to the autopilot gains. Extensive empirical numerical simulations are performed
to validate the proposed computational control algorithm.
