  Human input has enabled autonomous systems to improve their capabilities and
achieve complex behaviors that are otherwise challenging to generate
automatically. Recent work focuses on how robots can use such input - like
demonstrations or corrections - to learn intended objectives. These techniques
assume that the human's desired objective already exists within the robot's
hypothesis space. In reality, this assumption is often inaccurate: there will
always be situations where the person might care about aspects of the task that
the robot does not know about. Without this knowledge, the robot cannot infer
the correct objective. Hence, when the robot's hypothesis space is
misspecified, even methods that keep track of uncertainty over the objective
fail because they reason about which hypothesis might be correct, and not
whether any of the hypotheses are correct. In this paper, we posit that the
robot should reason explicitly about how well it can explain human inputs given
its hypothesis space and use that situational confidence to inform how it
should incorporate human input. We demonstrate our method on a 7
degree-of-freedom robot manipulator in learning from two important types of
human input: demonstrations of manipulation tasks, and physical corrections
during the robot's task execution.
