  One main obstacle for the wide use of deep learning in medical and
engineering sciences is its interpretability. While neural network models are
strong tools for making predictions, they often provide little information
about which features play significant roles in influencing the prediction
accuracy. To overcome this issue, many regularization procedures for learning
with neural networks have been proposed for dropping non-significant features.
Unfortunately, the lack of theoretical results casts doubt on the applicability
of such pipelines. In this work, we propose and establish a theoretical
guarantee for the use of the adaptive group lasso for selecting important
features of neural networks. Specifically, we show that our feature selection
method is consistent for single-output feed-forward neural networks with one
hidden layer and hyperbolic tangent activation function. We demonstrate its
applicability using both simulation and data analysis.
