  A long-term goal of reinforcement learning is to design agents that can
autonomously interact and learn in the world. A critical challenge to such
autonomy is the presence of irreversible states which require external
assistance to recover from, such as when a robot arm has pushed an object off
of a table. While standard agents require constant monitoring to decide when to
intervene, we aim to design proactive agents that can request human
intervention only when needed. To this end, we propose an algorithm that
efficiently learns to detect and avoid states that are irreversible, and
proactively asks for help in case the agent does enter them. On a suite of
continuous control environments with unknown irreversible states, we find that
our algorithm exhibits better sample- and intervention-efficiency compared to
existing methods. Our code is publicly available at
https://sites.google.com/view/proactive-interventions
