  This paper presents an observer-integrated Reinforcement Learning (RL)
approach, called Disturbance OBserver Network (DOB-Net), for robots operating
in environments where disturbances are unknown and time-varying, and may
frequently exceed robot control capabilities. The DOB-Net integrates a
disturbance dynamics observer network and a controller network. Originated from
conventional DOB mechanisms, the observer is built and enhanced via Recurrent
Neural Networks (RNNs), encoding estimation of past values and prediction of
future values of unknown disturbances in RNN hidden state. Such encoding allows
the controller generate optimal control signals to actively reject
disturbances, under the constraints of robot control capabilities. The observer
and the controller are jointly learned within policy optimization by advantage
actor critic. Numerical simulations on position regulation tasks have
demonstrated that the proposed DOB-Net significantly outperforms a conventional
feedback controller and classical RL algorithms.
