  In this paper, we rigorously derive Central Limit Theorems (CLT) for Bayesian
two-layerneural networks in the infinite-width limit and trained by variational
inference on a regression task. The different networks are trained via
different maximization schemes of the regularized evidence lower bound: (i) the
idealized case with exact estimation of a multiple Gaussian integral from the
reparametrization trick, (ii) a minibatch scheme using Monte Carlo sampling,
commonly known as Bayes-by-Backprop, and (iii) a computationally cheaper
algorithm named Minimal VI. The latter was recently introduced by leveraging
the information obtained at the level of the mean-field limit. Laws of large
numbers are already rigorously proven for the three schemes that admits the
same asymptotic limit. By deriving CLT, this work shows that the idealized and
Bayes-by-Backprop schemes have similar fluctuation behavior, that is different
from the Minimal VI one. Numerical experiments then illustrate that the Minimal
VI scheme is still more efficient, in spite of bigger variances, thanks to its
important gain in computational complexity.
