  When a human supervisor collaborates with a team of robots, their attention
is divided and cognitive resources are at a premium. We aim to optimize the
distribution of these resources and the flow of attention. To this end, we
propose the model of an idealized supervisor to describe human behavior. Such a
supervisor employs a potentially inaccurate internal model of the the robots'
dynamics to judge safety. We represent these safety judgements by constructing
a safe set from this internal model using reachability theory. When a robot
leaves this safe set, the idealized supervisor will intervene to assist,
regardless of whether or not the robot remains objectively safe. False
positives, where a human supervisor incorrectly judges a robot to be in danger,
needlessly consume supervisor attention. In this work, we propose a method that
decreases false positives by learning the supervisor's safe set and using that
information to govern robot behavior. We prove that robots behaving according
to our approach will reduce the occurrence of false positives for our idealized
supervisor model. Furthermore, we empirically validate our approach with a user
study that demonstrates a significant ($p = 0.0328$) reduction in false
positives for our method compared to a baseline safety controller.
