  This paper proposes to learn high-performance deep ConvNets with sparse
neural connections, referred to as sparse ConvNets, for face recognition. The
sparse ConvNets are learned in an iterative way, each time one additional layer
is sparsified and the entire model is re-trained given the initial weights
learned in previous iterations. One important finding is that directly training
the sparse ConvNet from scratch failed to find good solutions for face
recognition, while using a previously learned denser model to properly
initialize a sparser model is critical to continue learning effective features
for face recognition. This paper also proposes a new neural correlation-based
weight selection criterion and empirically verifies its effectiveness in
selecting informative connections from previously learned models in each
iteration. When taking a moderately sparse structure (26%-76% of weights in the
dense model), the proposed sparse ConvNet model significantly improves the face
recognition performance of the previous state-of-the-art DeepID2+ models given
the same training data, while it keeps the performance of the baseline model
with only 12% of the original parameters.
