  A Restricted Boltzmann Machine (RBM) is an unsupervised machine-learning
bipartite graphical model that jointly learns a probability distribution over
data and extracts their relevant statistical features. As such, RBM were
recently proposed for characterizing the patterns of coevolution between amino
acids in protein sequences and for designing new sequences. Here, we study how
the nature of the features learned by RBM changes with its defining parameters,
such as the dimensionality of the representations (size of the hidden layer)
and the sparsity of the features. We show that for adequate values of these
parameters, RBM operate in a so-called compositional phase in which visible
configurations sampled from the RBM are obtained by recombining these features.
We then compare the performance of RBM with other standard representation
learning algorithms, including Principal or Independent Component Analysis,
autoencoders (AE), variational auto-encoders (VAE), and their sparse variants.
We show that RBM, due to the stochastic mapping between data configurations and
representations, better capture the underlying interactions in the system and
are significantly more robust with respect to sample size than deterministic
methods such as PCA or ICA. In addition, this stochastic mapping is not
prescribed a priori as in VAE, but learned from data, which allows RBM to show
good performance even with shallow architectures. All numerical results are
illustrated on synthetic lattice-protein data, that share similar statistical
features with real protein sequences, and for which ground-truth interactions
are known.
