  "Deep Learning" methods attempt to learn generic features in an unsupervised
fashion from a large unlabelled data set. These generic features should perform
as well as the best hand crafted features for any learning problem that makes
use of this data. We provide a definition of generic features, characterize
when it is possible to learn them and provide methods closely related to the
autoencoder and deep belief network of deep learning. In order to do so we use
the notion of deficiency and illustrate its value in studying certain general
learning problems.
