  Defining sound and complete specifications for robots using formal languages
is challenging, while learning formal specifications directly from
demonstrations can lead to over-constrained task policies. In this paper, we
propose a Bayesian interactive robot training framework that allows the robot
to learn from both demonstrations provided by a teacher, and that teacher's
assessments of the robot's task executions. We also present an active learning
approach -- inspired by uncertainty sampling -- to identify the task execution
with the most uncertain degree of acceptability. Through a simulated
experiment, we demonstrate that our active learning approach identifies a
teacher's intended task specification with an equivalent or greater similarity
when compared to an approach that learns purely from demonstrations. Finally,
we demonstrate the efficacy of our approach in a real-world setting through a
user-study based on teaching a robot to set a dinner table.
