  A key characteristic of work on deep learning and neural networks in general
is that it relies on representations of the input that support generalization,
robust inference, domain adaptation and other desirable functionalities. Much
recent progress in the field has focused on efficient and effective methods for
computing representations. In this paper, we propose an alternative method that
is more efficient than prior work and produces representations that have a
property we call focality -- a property we hypothesize to be important for
neural network representations. The method consists of a simple application of
two consecutive SVDs and is inspired by Anandkumar (2012).
