  Parkour is a grand challenge for legged locomotion that requires robots to
overcome various obstacles rapidly in complex environments. Existing methods
can generate either diverse but blind locomotion skills or vision-based but
specialized skills by using reference animal data or complex rewards. However,
autonomous parkour requires robots to learn generalizable skills that are both
vision-based and diverse to perceive and react to various scenarios. In this
work, we propose a system for learning a single end-to-end vision-based parkour
policy of diverse parkour skills using a simple reward without any reference
motion data. We develop a reinforcement learning method inspired by direct
collocation to generate parkour skills, including climbing over high obstacles,
leaping over large gaps, crawling beneath low barriers, squeezing through thin
slits, and running. We distill these skills into a single vision-based parkour
policy and transfer it to a quadrupedal robot using its egocentric depth
camera. We demonstrate that our system can empower two different low-cost
robots to autonomously select and execute appropriate parkour skills to
traverse challenging real-world environments.
