  Stable gait generation is a crucial problem for legged robot locomotion as
this impacts other critical performance factors such as, e.g. mobility over an
uneven terrain and power consumption. Gait generation stability results from
the efficient control of the interaction between the legged robot's body and
the environment where it moves. Here, we study how this can be achieved by a
combination of model-predictive and predictive reinforcement learning
controllers. Model-predictive control (MPC) is a well-established method that
does not utilize any online learning (except for some adaptive variations) as
it provides a convenient interface for state constraints management.
Reinforcement learning (RL), in contrast, relies on adaptation based on pure
experience. In its bare-bone variants, RL is not always suitable for robots due
to their high complexity and expensive simulation/experimentation. In this
work, we combine both control methods to address the quadrupedal robot stable
gate generation problem. The hybrid approach that we develop and apply uses a
cost roll-out algorithm with a tail cost in the form of a Q-function modeled by
a neural network; this allows to alleviate the computational complexity, which
grows exponentially with the prediction horizon in a purely MPC approach. We
demonstrate that our RL gait controller achieves stable locomotion at short
horizons, where a nominal MP controller fails. Further, our controller is
capable of live operation, meaning that it does not require previous training.
Our results suggest that the hybridization of MPC with RL, as presented here,
is beneficial to achieve a good balance between online control capabilities and
computational complexity.
