  Reinforcement learning (RL) algorithms hold the promise of enabling
autonomous skill acquisition for robotic systems. However, in practice,
real-world robotic RL typically requires time consuming data collection and
frequent human intervention to reset the environment. Moreover, robotic
policies learned with RL often fail when deployed beyond the carefully
controlled setting in which they were learned. In this work, we study how these
challenges can all be tackled by effective utilization of diverse offline
datasets collected from previously seen tasks. When faced with a new task, our
system adapts previously learned skills to quickly learn to both perform the
new task and return the environment to an initial state, effectively performing
its own environment reset. Our empirical results demonstrate that incorporating
prior data into robotic reinforcement learning enables autonomous learning,
substantially improves sample-efficiency of learning, and enables better
generalization. Project website: https://sites.google.com/view/ariel-berkeley/
