  Slack and margin rescaling are variants of the structured output SVM, which
is frequently applied to problems in computer vision such as image
segmentation, object localization, and learning parts based object models. They
define convex surrogates to task specific loss functions, which, when
specialized to non-additive loss functions for multi-label problems, yield
extensions to increasing set functions. We demonstrate in this paper that we
may use these concepts to define polynomial time convex extensions of arbitrary
supermodular functions, providing an analysis framework for the tightness of
these surrogates. This analysis framework shows that, while neither margin nor
slack rescaling dominate the other, known bounds on supermodular functions can
be used to derive extensions that dominate both of these, indicating possible
directions for defining novel structured output prediction surrogates. In
addition to the analysis of structured prediction loss functions, these results
imply an approach to supermodular minimization in which margin rescaling is
combined with non-polynomial time convex extensions to compute a sequence of LP
relaxations reminiscent of a cutting plane method. This approach is applied to
the problem of selecting representative exemplars from a set of images,
validating our theoretical contributions.
