  Optimizing with group sparsity is significant in enhancing model
interpretability in machining learning applications, e.g., feature selection,
compressed sensing and model compression. However, for large-scale stochastic
training problems, effective group sparsity exploration are typically hard to
achieve. Particularly, the state-of-the-art stochastic optimization algorithms
usually generate merely dense solutions. To overcome this shortage, we propose
a stochastic method -- Half-space Stochastic Projected Gradient (HSPG) method
to search solutions of high group sparsity while maintain the convergence.
Initialized by a simple Prox-SG Step, the HSPG method relies on a novel
Half-Space Step to substantially boost the sparsity level. Numerically, HSPG
demonstrates its superiority in deep neural networks, e.g., VGG16, ResNet18 and
MobileNetV1, by computing solutions of higher group sparsity, competitive
objective values and generalization accuracy.
