  Recent advances in legged locomotion have enabled quadrupeds to walk on
challenging terrains. However, bipedal robots are inherently more unstable and
hence it's harder to design walking controllers for them. In this work, we
leverage recent advances in rapid adaptation for locomotion control, and extend
them to work on bipedal robots. Similar to existing works, we start with a base
policy which produces actions while taking as input an estimated extrinsics
vector from an adaptation module. This extrinsics vector contains information
about the environment and enables the walking controller to rapidly adapt
online. However, the extrinsics estimator could be imperfect, which might lead
to poor performance of the base policy which expects a perfect estimator. In
this paper, we propose A-RMA (Adapting RMA), which additionally adapts the base
policy for the imperfect extrinsics estimator by finetuning it using model-free
RL. We demonstrate that A-RMA outperforms a number of RL-based baseline
controllers and model-based controllers in simulation, and show zero-shot
deployment of a single A-RMA policy to enable a bipedal robot, Cassie, to walk
in a variety of different scenarios in the real world beyond what it has seen
during training. Videos and results at https://ashish-kmr.github.io/a-rma/
