  In this letter, we present a method for integrating central pattern
generators (CPGs), i.e. systems of coupled oscillators, into the deep
reinforcement learning (DRL) framework to produce robust and omnidirectional
quadruped locomotion. The agent learns to directly modulate the intrinsic
oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior
among different oscillators. This approach also allows the use of DRL to
explore questions related to neuroscience, namely the role of descending
pathways, interoscillator couplings, and sensory feedback in gait generation.
We train our policies in simulation and perform a sim-to-real transfer to the
Unitree A1 quadruped, where we observe robust behavior to disturbances unseen
during training, most notably to a dynamically added 13.75 kg load representing
115% of the nominal quadruped mass. We test several different observation
spaces based on proprioceptive sensing and show that our framework is
deployable with no domain randomization and very little feedback, where along
with the oscillator states, it is possible to provide only contact booleans in
the observation space. Video results can be found at
https://youtu.be/xqXHLzLsEV4.
