  To proactively navigate and traverse various terrains, active use of visual
perception becomes indispensable. We aim to investigate the feasibility and
performance of using sparse visual observations to achieve perceptual
locomotion over a range of common terrains (steps, ramps, gaps, and stairs) in
human-centered environments. We formulate a selection of sparse visual inputs
suitable for locomotion over the terrains of interest, and propose a learning
framework to integrate exteroceptive and proprioceptive states. We specifically
design the state observations and a training curriculum to learn feedback
control policies effectively over a range of different terrains. We extensively
validate and benchmark the learned policy in various tasks: omnidirectional
walking on flat ground, and forward locomotion over various obstacles, showing
high success rate of traversability. Furthermore, we study exteroceptive
ablations and evaluate policy generalization by adding various levels of noise
and testing on new unseen terrains. We demonstrate the capabilities of
autonomous perceptual locomotion that can be achieved by only using sparse
visual observations from direct depth measurements, which are easily available
from a Lidar or RGB-D sensor, showing robust ascent and descent over high
stairs of 20 cm height, i.e., 50% leg length, and robustness against noise and
unseen terrains.
