  Whereas CNNs have demonstrated immense progress in many vision problems, they
suffer from a dependence on monumental amounts of labeled training data. On the
other hand, dictionary learning does not scale to the size of problems that
CNNs can handle, despite being very effective at low-level vision tasks such as
denoising and inpainting. Recently, interest has grown in adapting dictionary
learning methods for supervised tasks such as classification and inverse
problems. We propose two new network layers that are based on dictionary
learning: a sparse factorization layer and a convolutional sparse factorization
layer, analogous to fully-connected and convolutional layers, respectively.
Using our derivations, these layers can be dropped in to existing CNNs, trained
together in an end-to-end fashion with back-propagation, and leverage
semisupervision in ways classical CNNs cannot. We experimentally compare
networks with these two new layers against a baseline CNN. Our results
demonstrate that networks with either of the sparse factorization layers are
able to outperform classical CNNs when supervised data are few. They also show
performance improvements in certain tasks when compared to the CNN with no
sparse factorization layers with the same exact number of parameters.
