  This paper addresses the problem of legged locomotion in non-flat terrain. As
legged robots such as quadrupeds are to be deployed in terrains with geometries
which are difficult to model and predict, the need arises to equip them with
the capability to generalize well to unforeseen situations. In this work, we
propose a novel technique for training neural-network policies for
terrain-aware locomotion, which combines state-of-the-art methods for
model-based motion planning and reinforcement learning. Our approach is
centered on formulating Markov decision processes using the evaluation of
dynamic feasibility criteria in place of physical simulation. We thus employ
policy-gradient methods to independently train policies which respectively plan
and execute foothold and base motions in 3D environments using both
proprioceptive and exteroceptive measurements. We apply our method within a
challenging suite of simulated terrain scenarios which contain features such as
narrow bridges, gaps and stepping-stones, and train policies which succeed in
locomoting effectively in all cases.
