  We propose a novel hierarchical generative model with a simple Markovian
structure and a corresponding inference model. Both the generative and
inference model are trained using the adversarial learning paradigm. We
demonstrate that the hierarchical structure supports the learning of
progressively more abstract representations as well as providing semantically
meaningful reconstructions with different levels of fidelity. Furthermore, we
show that minimizing the Jensen-Shanon divergence between the generative and
inference network is enough to minimize the reconstruction error. The resulting
semantically meaningful hierarchical latent structure discovery is exemplified
on the CelebA dataset. There, we show that the features learned by our model in
an unsupervised way outperform the best handcrafted features. Furthermore, the
extracted features remain competitive when compared to several recent deep
supervised approaches on an attribute prediction task on CelebA. Finally, we
leverage the model's inference network to achieve state-of-the-art performance
on a semi-supervised variant of the MNIST digit classification task.
