  We extend the learning from demonstration paradigm by providing a method for
learning unknown constraints shared across tasks, using demonstrations of the
tasks, their cost functions, and knowledge of the system dynamics and control
constraints. Given safe demonstrations, our method uses hit-and-run sampling to
obtain lower cost, and thus unsafe, trajectories. Both safe and unsafe
trajectories are used to obtain a consistent representation of the unsafe set
via solving an integer program. Our method generalizes across system dynamics
and learns a guaranteed subset of the constraint. We also provide theoretical
analysis on what subset of the constraint can be learnable from safe
demonstrations. We demonstrate our method on linear and nonlinear system
dynamics, show that it can be modified to work with suboptimal demonstrations,
and that it can also be used to learn constraints in a feature space.
