  We present an Imitation Learning approach for the control of dynamical
systems with a known model. Our policy search method is guided by solutions
from MPC. Typical policy search methods of this kind minimize a distance metric
between the guiding demonstrations and the learned policy. Our loss function,
however, corresponds to the minimization of the control Hamiltonian, which
derives from the principle of optimality. Therefore, our algorithm directly
attempts to solve the optimality conditions with a parameterized class of
control laws. Additionally, the proposed loss function explicitly encodes the
constraints of the optimal control problem and we provide numerical evidence
that its minimization achieves improved constraint satisfaction. We train a
mixture-of-expert neural network architecture for controlling a quadrupedal
robot and show that this policy structure is well suited for such multimodal
systems. The learned policy can successfully stabilize different gaits on the
real walking robot from less than 10 min of demonstration data.
