  Recent advances in deep learning and neural networks have led to an increased
interest in the application of generative models in statistical and condensed
matter physics. In particular, restricted Boltzmann machines (RBMs) and
variational autoencoders (VAEs) as specific classes of neural networks have
been successfully applied in the context of physical feature extraction and
representation learning. Despite these successes, however, there is only
limited understanding of their representational properties and limitations. To
better understand the representational characteristics of RBMs and VAEs, we
study their ability to capture physical features of the Ising model at
different temperatures. This approach allows us to quantitatively assess
learned representations by comparing sample features with corresponding
theoretical predictions. Our results suggest that the considered RBMs and
convolutional VAEs are able to capture the temperature dependence of
magnetization, energy, and spin-spin correlations. The samples generated by
RBMs are more evenly distributed across temperature than those generated by
VAEs. We also find that convolutional layers in VAEs are important to model
spin correlations whereas RBMs achieve similar or even better performances
without convolutional filters.
