  A restricted Boltzmann machine is a generative probabilistic graphic network.
A probability of finding the network in a certain configuration is given by the
Boltzmann distribution. Given training data, its learning is done by optimizing
parameters of the energy function of the network. In this paper, we analyze the
training process of the restricted Boltzmann machine in the context of
statistical physics. As an illustration, for small size Bar-and-Stripe
patterns, we calculate thermodynamic quantities such as entropy, free energy,
and internal energy as a function of training epoch. We demonstrate the growth
of the correlation between the visible and hidden layers via the subadditivity
of entropies as the training proceeds. Using the Monte-Carlo simulation of
trajectories of the visible and hidden vectors in configuration space, we also
calculate the distribution of the work done on the restricted Boltzmann machine
by switching the parameters of the energy function. We discuss the Jarzynski
equality which connects the path average of the exponential function of the
work and the difference in free energies before and after training.
