  We introduce a simple recurrent variational auto-encoder architecture that
significantly improves image modeling. The system represents the
state-of-the-art in latent variable models for both the ImageNet and Omniglot
datasets. We show that it naturally separates global conceptual information
from lower level details, thus addressing one of the fundamentally desired
properties of unsupervised learning. Furthermore, the possibility of
restricting ourselves to storing only global information about an image allows
us to achieve high quality 'conceptual compression'.
