  This paper contributes a first study into how different human users deliver
simultaneous control and feedback signals during human-robot interaction. As
part of this work, we formalize and present a general interactive learning
framework for online cooperation between humans and reinforcement learning
agents. In many human-machine interaction settings, there is a growing gap
between the degrees-of-freedom of complex semi-autonomous systems and the
number of human control channels. Simple human control and feedback mechanisms
are required to close this gap and allow for better collaboration between
humans and machines on complex tasks. To better inform the design of concurrent
control and feedback interfaces, we present experimental results from a
human-robot collaborative domain wherein the human must simultaneously deliver
both control and feedback signals to interactively train an actor-critic
reinforcement learning robot. We compare three experimental conditions: 1)
human delivered control signals, 2) reward-shaping feedback signals, and 3)
simultaneous control and feedback. Our results suggest that subjects provide
less feedback when simultaneously delivering feedback and control signals and
that control signal quality is not significantly diminished. Our data suggest
that subjects may also modify when and how they provide feedback. Through
algorithmic development and tuning informed by this study, we expect
semi-autonomous actions of robotic agents can be better shaped by human
feedback, allowing for seamless collaboration and improved performance in
difficult interactive domains.
