  We discuss different types of human-robot interaction paradigms in the
context of training end-to-end reinforcement learning algorithms. We provide a
taxonomy to categorize the types of human interaction and present our
Cycle-of-Learning framework for autonomous systems that combines different
human-interaction modalities with reinforcement learning. Two key concepts
provided by our Cycle-of-Learning framework are how it handles the integration
of the different human-interaction modalities (demonstration, intervention, and
evaluation) and how to define the switching criteria between them.
