  In this paper, we propose embedding sparsity into the structure of deep
neural networks, where model parameters can be exactly zero during training
with the stochastic gradient descent. Thus, it can learn the sparsified
structure and the weights of networks simultaneously. The proposed approach can
learn structured as well as unstructured sparsity.
