  Deep learning has powered recent successes of artificial intelligence (AI).
However, the deep neural network, as the basic model of deep learning, has
suffered from issues such as local traps and miscalibration. In this paper, we
provide a new framework for sparse deep learning, which has the above issues
addressed in a coherent way. In particular, we lay down a theoretical
foundation for sparse deep learning and propose prior annealing algorithms for
learning sparse neural networks. The former has successfully tamed the sparse
deep neural network into the framework of statistical modeling, enabling
prediction uncertainty correctly quantified. The latter can be asymptotically
guaranteed to converge to the global optimum, enabling the validity of the
down-stream statistical inference. Numerical result indicates the superiority
of the proposed method compared to the existing ones.
