  The success of deep learning has been due, in no small part, to the
availability of large annotated datasets. Thus, a major bottleneck in current
learning pipelines is the time-consuming human annotation of data. In scenarios
where such input-output pairs cannot be collected, simulation is often used
instead, leading to a domain-shift between synthesized and real-world data.
This work offers an unsupervised alternative that relies on the availability of
task-specific energy functions, replacing the generic supervised loss. Such
energy functions are assumed to lead to the desired label as their minimizer
given the input. The proposed approach, termed "Deep Energy", trains a Deep
Neural Network (DNN) to approximate this minimization for any chosen input.
Once trained, a simple and fast feed-forward computation provides the inferred
label. This approach allows us to perform unsupervised training of DNNs with
real-world inputs only, and without the need for manually-annotated labels, nor
synthetically created data. "Deep Energy" is demonstrated in this paper on
three different tasks -- seeded segmentation, image matting and single image
dehazing -- exposing its generality and wide applicability. Our experiments
show that the solution provided by the network is often much better in quality
than the one obtained by a direct minimization of the energy function,
suggesting an added regularization property in our scheme.
