  To achieve successful field autonomy, mobile robots need to freely adapt to
changes in their environment. Visual navigation systems such as Visual Teach
and Repeat (VT&R) often assume the space around the reference trajectory is
free, but if the environment is obstructed path tracking can fail or the robot
could collide with a previously unseen obstacle. In this work, we present a
locally reactive controller for a VT&R system that allows a robot to navigate
safely despite physical changes to the environment. Our controller uses a local
elevation map to compute vector representations and outputs twist commands for
navigation at 10 Hz. They are combined in a Riemannian Motion Policies (RMP)
controller that requires <2 ms to run on a CPU. We integrated our controller
with a VT&R system onboard an ANYmal C robot and tested it in indoor cluttered
spaces and a large-scale underground mine. We demonstrate that our locally
reactive controller keeps the robot safe when physical occlusions or loss of
visual tracking occur such as when walking close to walls, crossing doorways,
or traversing narrow corridors. Video: https://youtu.be/G_AwNec5AwU
