  Humans can perform parkour by traversing obstacles in a highly dynamic
fashion requiring precise eye-muscle coordination and movement. Getting robots
to do the same task requires overcoming similar challenges. Classically, this
is done by independently engineering perception, actuation, and control systems
to very low tolerances. This restricts them to tightly controlled settings such
as a predetermined obstacle course in labs. In contrast, humans are able to
learn parkour through practice without significantly changing their underlying
biology. In this paper, we take a similar approach to developing robot parkour
on a small low-cost robot with imprecise actuation and a single front-facing
depth camera for perception which is low-frequency, jittery, and prone to
artifacts. We show how a single neural net policy operating directly from a
camera image, trained in simulation with large-scale RL, can overcome imprecise
sensing and actuation to output highly precise control behavior end-to-end. We
show our robot can perform a high jump on obstacles 2x its height, long jump
across gaps 2x its length, do a handstand and run across tilted ramps, and
generalize to novel obstacle courses with different physical properties.
Parkour videos at https://extreme-parkour.github.io/
