  Constraint-based learning reduces the burden of collecting labels by having
users specify general properties of structured outputs, such as constraints
imposed by physical laws. We propose a novel framework for simultaneously
learning these constraints and using them for supervision, bypassing the
difficulty of using domain expertise to manually specify constraints. Learning
requires a black-box simulator of structured outputs, which generates valid
labels, but need not model their corresponding inputs or the input-label
relationship. At training time, we constrain the model to produce outputs that
cannot be distinguished from simulated labels by adversarial training.
Providing our framework with a small number of labeled inputs gives rise to a
new semi-supervised structured prediction model; we evaluate this model on
multiple tasks --- tracking, pose estimation and time series prediction --- and
find that it achieves high accuracy with only a small number of labeled inputs.
In some cases, no labels are required at all.
