  Structured Prediction Energy Networks (SPENs) are a simple, yet expressive
family of structured prediction models (Belanger and McCallum, 2016). An energy
function over candidate structured outputs is given by a deep network, and
predictions are formed by gradient-based optimization. This paper presents
end-to-end learning for SPENs, where the energy function is discriminatively
trained by back-propagating through gradient-based prediction. In our
experience, the approach is substantially more accurate than the structured SVM
method of Belanger and McCallum (2016), as it allows us to use more
sophisticated non-convex energies. We provide a collection of techniques for
improving the speed, accuracy, and memory requirements of end-to-end SPENs, and
demonstrate the power of our method on 7-Scenes image denoising and CoNLL-2005
semantic role labeling tasks. In both, inexact minimization of non-convex SPEN
energies is superior to baseline methods that use simplistic energy functions
that can be minimized exactly.
