  Restricted Boltzmann Machines are simple and powerful generative models that
can encode any complex dataset. Despite all their advantages, in practice the
trainings are often unstable and it is difficult to assess their quality
because the dynamics are affected by extremely slow time dependencies. This
situation becomes critical when dealing with low-dimensional clustered
datasets, where the time required to sample ergodically the trained models
becomes computationally prohibitive. In this work, we show that this divergence
of Monte Carlo mixing times is related to a phenomenon of phase coexistence,
similar to that which occurs in physics near a first-order phase transition. We
show that sampling the equilibrium distribution using the Markov chain Monte
Carlo method can be dramatically accelerated when using biased sampling
techniques, in particular the Tethered Monte Carlo (TMC) method. This sampling
technique efficiently solves the problem of evaluating the quality of a given
trained model and generating new samples in a reasonable amount of time.
Moreover, we show that this sampling technique can also be used to improve the
computation of the log-likelihood gradient during training, leading to dramatic
improvements in training RBMs with artificial clustered datasets. On real
low-dimensional datasets, this new training method fits RBM models with
significantly faster relaxation dynamics than those obtained with standard PCD
recipes. We also show that TMC sampling can be used to recover the free-energy
profile of the RBM. This proves to be extremely useful to compute the
probability distribution of a given model and to improve the generation of new
decorrelated samples in slow PCD-trained models.
