  Despite the popularity of deep learning, structure learning for deep models
remains a relatively under-explored area. In contrast, structure learning has
been studied extensively for probabilistic graphical models (PGMs). In
particular, an efficient algorithm has been developed for learning a class of
tree-structured PGMs called hierarchical latent tree models (HLTMs), where
there is a layer of observed variables at the bottom and multiple layers of
latent variables on top. In this paper, we propose a simple method for learning
the structures of feedforward neural networks (FNNs) based on HLTMs. The idea
is to expand the connections in the tree skeletons from HLTMs and to use the
resulting structures for FNNs. An important characteristic of FNN structures
learned this way is that they are sparse. We present extensive empirical
results to show that, compared with standard FNNs tuned-manually, sparse FNNs
learned by our method achieve better or comparable classification performance
with much fewer parameters. They are also more interpretable.
