  There have been attempts in reinforcement learning to exploit a priori
knowledge about the structure of the system. This paper proposes a hybrid
reinforcement learning controller which dynamically interpolates a model-based
linear controller and an arbitrary differentiable policy. The linear controller
is designed based on local linearised model knowledge, and stabilises the
system in a neighbourhood about an operating point. The coefficients of
interpolation between the two controllers are determined by a scaled distance
function measuring the distance between the current state and the operating
point. The overall hybrid controller is proven to maintain the stability
guarantee around the neighborhood of the operating point and still possess the
universal function approximation property of the arbitrary non-linear policy.
Learning has been done on both model-based (PILCO) and model-free (DDPG)
frameworks. Simulation experiments performed in OpenAI gym demonstrate
stability and robustness of the proposed hybrid controller. This paper thus
introduces a principled method allowing for the direct importing of control
methodology into reinforcement learning.
