  Underpinning the success of deep learning is effective regularizations that
allow a variety of priors in data to be modeled. For example, robustness to
adversarial perturbations, and correlations between multiple modalities.
However, most regularizers are specified in terms of hidden layer outputs,
which are not themselves optimization variables. In contrast to prevalent
methods that optimize them indirectly through model weights, we propose
inserting proximal mapping as a new layer to the deep network, which directly
and explicitly produces well regularized hidden layer outputs. The resulting
technique is shown well connected to kernel warping and dropout, and novel
algorithms were developed for robust temporal learning and multiview modeling,
both outperforming state-of-the-art methods.
