  To succeed in the real world, robots must cope with situations that differ
from those seen during training. We study the problem of adapting on-the-fly to
such novel scenarios during deployment, by drawing upon a diverse repertoire of
previously learned behaviors. Our approach, RObust Autonomous Modulation
(ROAM), introduces a mechanism based on the perceived value of pre-trained
behaviors to select and adapt pre-trained behaviors to the situation at hand.
Crucially, this adaptation process all happens within a single episode at test
time, without any human supervision. We provide theoretical analysis of our
selection mechanism and demonstrate that ROAM enables a robot to adapt rapidly
to changes in dynamics both in simulation and on a real Go1 quadruped, even
successfully moving forward with roller skates on its feet. Our approach adapts
over 2x as efficiently compared to existing methods when facing a variety of
out-of-distribution situations during deployment by effectively choosing and
adapting relevant behaviors on-the-fly.
