  Quadruped animals are capable of exhibiting a diverse range of locomotion
gaits. While progress has been made in demonstrating such gaits on robots,
current methods rely on motion priors, dynamics models, or other forms of
extensive manual efforts. People can use natural language to describe dance
moves. Could one use a formal language to specify quadruped gaits? To this end,
we aim to enable easy gait specification and efficient policy learning.
Leveraging Reward Machines~(RMs) for high-level gait specification over foot
contacts, our approach is called RM-based Locomotion Learning~(RMLL), and
supports adjusting gait frequency at execution time. Gait specification is
enabled through the use of a few logical rules per gait (e.g., alternate
between moving front feet and back feet) and does not require labor-intensive
motion priors. Experimental results in simulation highlight the diversity of
learned gaits (including two novel gaits), their energy consumption and
stability across different terrains, and the superior sample-efficiency when
compared to baselines. We also demonstrate these learned policies with a real
quadruped robot. Video and supplementary materials:
https://sites.google.com/view/rm-locomotion-learning/home
