  This paper describes an online off-policy data-driven reinforcement learning
based-algorithm to regulate and control the relative position of a deputy
satellite in an autonomous satellite docking problem. The optimal control
policy is learned under the framework of output regulation problem and adaptive
dynamic programming (ADP) by considering the continuous-time linearized model
of the satellite. The linearized model of relative motion is used to describe
the motion between satellites, and the satellite docking problem is formulated
as a linear optimal output regulation problem, in which the feedback-forward
optimal controller is used to track a class of references and rejecting a class
of disturbances while maintaining the overall system's closed-loop stability.
The optimal control problem is presented using a data-driven reinforcement
learning based method to regulate the relative position and velocity of the
deputy to safely dock with the chief. Using the adaptive optimal output
regulation framework, the learned optimal feedback-feedforward gains guarantee
optimal transient and steady state performances without any prior knowledge of
the dynamics of the studied system. {The states/input information of the
underlying dynamical system are instead used to compute the approximated
optimal feedback-feedforward control gain matrices.} Reference tracking and
disturbance rejection are achieved in an optimal sense without using any
modelling information of the physics of the satellites. {Simulation results are
presented and demonstrate the efficacy of the proposed method.
