  We present a theoretical analysis of Gaussian-binary restricted Boltzmann
machines (GRBMs) from the perspective of density models. The key aspect of this
analysis is to show that GRBMs can be formulated as a constrained mixture of
Gaussians, which gives a much better insight into the model's capabilities and
limitations. We show that GRBMs are capable of learning meaningful features
both in a two-dimensional blind source separation task and in modeling natural
images. Further, we show that reported difficulties in training GRBMs are due
to the failure of the training algorithm rather than the model itself. Based on
our analysis we are able to propose several training recipes, which allowed
successful and fast training in our experiments. Finally, we discuss the
relationship of GRBMs to several modifications that have been proposed to
improve the model.
