  Recently, work on reinforcement learning (RL) for bipedal robots has
successfully learned controllers for a variety of dynamic gaits with robust
sim-to-real demonstrations. In order to maintain balance, the learned
controllers have full freedom of where to place the feet, resulting in highly
robust gaits. In the real world however, the environment will often impose
constraints on the feasible footstep locations, typically identified by
perception systems. Unfortunately, most demonstrated RL controllers on bipedal
robots do not allow for specifying and responding to such constraints. This
missing control interface greatly limits the real-world application of current
RL controllers. In this paper, we aim to maintain the robust and dynamic nature
of learned gaits while also respecting footstep constraints imposed externally.
We develop an RL formulation for training dynamic gait controllers that can
respond to specified touchdown locations. We then successfully demonstrate
simulation and sim-to-real performance on the bipedal robot Cassie. In
addition, we use supervised learning to induce a transition model for
accurately predicting the next touchdown locations that the controller can
achieve given the robot's proprioceptive observations. This model paves the way
for integrating the learned controller into a full-order robot locomotion
planner that robustly satisfies both balance and environmental constraints.
