  In this paper, we describe the deep sparse coding network (SCN), a novel deep
network that encodes intermediate representations with nonnegative sparse
coding. The SCN is built upon a number of cascading bottleneck modules, where
each module consists of two sparse coding layers with relatively wide and slim
dictionaries that are specialized to produce high dimensional discriminative
features and low dimensional representations for clustering, respectively.
During training, both the dictionaries and regularization parameters are
optimized with an end-to-end supervised learning algorithm based on multilevel
optimization. Effectiveness of an SCN with seven bottleneck modules is verified
on several popular benchmark datasets. Remarkably, with few parameters to
learn, our SCN achieves 5.81% and 19.93% classification error rate on CIFAR-10
and CIFAR-100, respectively.
