  Boltzmann Machines (BMs) are graphical models with interconnected binary
units, employed for the unsupervised modeling of data distributions. When
trained on real data, BMs show the tendency to behave like critical systems,
displaying a high susceptibility of the model under a small rescaling of the
inferred parameters. This behaviour is not convenient for the purpose of
generating data, because it slows down the sampling process, and induces the
model to overfit the training-data. In this study, we introduce a
regularization method for BMs to improve the robustness of the model under
rescaling of the parameters. The new technique shares formal similarities with
the unlearning algorithm, an iterative procedure used to improve memory
associativity in Hopfield-like neural networks. We test our unlearning
regularization on synthetic data generated by two simple models, the
Curie-Weiss ferromagnetic model and the Sherrington-Kirkpatrick spin glass
model. We show that it outperforms $L_p$-norm schemes and discuss the role of
parameter initialization. Eventually, the method is applied to learn the
activity of real neuronal cells, confirming its efficacy at shifting the
inferred model away from criticality and coming out as a powerful candidate for
actual scientific implementations.
