  Mean field (MF) approximation offers a simple, fast way to infer direct
interactions between elements in a network of correlated variables, a common,
computationally challenging problem with practical applications in fields
ranging from physics and biology to the social sciences. However, MF methods
achieve their best performance with strong regularization, well beyond Bayesian
expectations, an empirical fact that is poorly understood. In this work, we
study the influence of pseudo-count and $L_2$-norm regularization schemes on
the quality of inferred Ising or Potts interaction networks from correlation
data within the MF approximation. We argue, based on the analysis of small
systems, that the optimal value of the regularization strength remains finite
even if the sampling noise tends to zero, in order to correct for systematic
biases introduced by the MF approximation. Our claim is corroborated by
extensive numerical studies of diverse model systems and by the analytical
study of the $m$-component spin model, for large but finite $m$. Additionally
we find that pseudo-count regularization is robust against sampling noise, and
often outperforms $L_2$-norm regularization, particularly when the underlying
network of interactions is strongly heterogeneous. Much better performances are
generally obtained for the Ising model than for the Potts model, for which only
couplings incoming onto medium-frequency symbols are reliably inferred.
