  Improved mean-field technics are a central theme of statistical physics
methods applied to inference and learning. We revisit here some of these
methods using high-temperature expansions for disordered systems initiated by
Plefka, Georges and Yedidia. We derive the Gibbs free entropy and the
subsequent self-consistent equations for a generic class of statistical models
with correlated matrices and show in particular that many classical
approximation schemes, such as adaptive TAP, Expectation-Consistency, or the
approximations behind the Vector Approximate Message Passing algorithm all rely
on the same assumptions, that are also at the heart of high-temperature
expansions. We focus on the case of rotationally invariant random coupling
matrices in the `high-dimensional' limit in which the number of samples and the
dimension are both large, but with a fixed ratio. This encapsulates many widely
studied models, such as Restricted Boltzmann Machines or Generalized Linear
Models with correlated data matrices. In this general setting, we show that all
the approximation schemes described before are equivalent, and we conjecture
that they are exact in the thermodynamic limit in the replica symmetric phases.
We achieve this conclusion by resummation of the infinite perturbation series,
which generalizes a seminal result of Parisi and Potters. A rigorous derivation
of this conjecture is an interesting mathematical challenge. On the way to
these conclusions, we uncover several diagrammatical results in connection with
free probability and random matrix theory, that are interesting independently
of the rest of our work.
