  The restricted Boltzmann machine (RBM) is a two-layer energy-based model that
uses its hidden-visible connections to learn the underlying distribution of
visible units, whose interactions are often complicated by high-order
correlations. Previous studies on the Ising model of small system sizes have
shown that RBMs are able to accurately learn the Boltzmann distribution and
reconstruct thermal quantities at temperatures away from the critical point
$T_c$. How the RBM encodes the Boltzmann distribution and captures the phase
transition are, however, not well explained. In this work, we perform RBM
learning of the $2d$ and $3d$ Ising model and carefully examine how the RBM
extracts useful probabilistic and physical information from Ising
configurations. We find several indicators derived from the weight matrix that
could characterize the Ising phase transition. We verify that the hidden
encoding of a visible state tends to have an equal number of positive and
negative units, whose sequence is randomly assigned during training and can be
inferred by analyzing the weight matrix. We also explore the physical meaning
of visible energy and loss function (pseudo-likelihood) of the RBM and show
that they could be harnessed to predict the critical point or estimate physical
quantities such as entropy.
