  Thanks to the explosive developments of data-driven learning methodologies
recently, reinforcement learning (RL) emerges as a promising solution to
address the legged locomotion problem in robotics. In this manuscript, we
propose a novel concurrent teacher-student reinforcement learning architecture
for legged locomotion over challenging terrains, based only on proprioceptive
measurements in real-world deployment. Different from convectional
teacher-student architecture that trains the teacher policy via RL and
transfers the knowledge to the student policy through supervised learning, our
proposed architecture trains teacher and student policy networks concurrently
under the reinforcement learning paradigm. To achieve this, we develop a new
training scheme based on conventional proximal policy gradient (PPO) method to
accommodate the interaction between teacher policy network and student policy
network. The effectiveness of the proposed architecture as well as the new
training scheme is demonstrated through extensive indoor and outdoor
experiments on quadrupedal robots and point-foot bipedal robot, showcasing
robust locomotion over challenging terrains and improved performance compared
to two-stage training methods.
