  Motivated by recent progress in using restricted Boltzmann machines as
preprocessing algorithms for deep neural network, we revisit the mean-field
equations (belief-propagation and TAP equations) in the best understood such
machine, namely the Hopfield model of neural networks, and we explicit how they
can be used as iterative message-passing algorithms, providing a fast method to
compute the local polarizations of neurons. In the "retrieval phase" where
neurons polarize in the direction of one memorized pattern, we point out a
major difference between the belief propagation and TAP equations : the set of
belief propagation equations depends on the pattern which is retrieved, while
one can use a unique set of TAP equations. This makes the latter method much
better suited for applications in the learning process of restricted Boltzmann
machines. In the case where the patterns memorized in the Hopfield model are
not independent, but are correlated through a combinatorial structure, we show
that the TAP equations have to be modified. This modification can be seen
either as an alteration of the reaction term in TAP equations, or, more
interestingly, as the consequence of message passing on a graphical model with
several hidden layers, where the number of hidden layers depends on the depth
of the correlations in the memorized patterns. This layered structure is
actually necessary when one deals with more general restricted Boltzmann
machines.
