  We study the validity of the Naive Mean Field (NMF) approximation for
canonical GLMs with product priors. This setting is challenging due to the
non-conjugacy of the likelihood and the prior. Using the theory of non-linear
large deviations (Austin 2019, Chatterjee, Dembo 2016, Eldan 2018), we derive
sufficient conditions for the tightness of the NMF approximation to the
log-normalizing constant of the posterior distribution. As a second
contribution, we establish that under minor conditions on the design, any NMF
optimizer is a product distribution where each component is a quadratic tilt of
the prior. In turn, this suggests novel iterative algorithms for fitting the
NMF optimizer to the target posterior. Finally, we establish that if the NMF
optimization problem has a "well-separated maximizer", then this optimizer
governs the probabilistic properties of the posterior. Specifically, we derive
credible intervals with average coverage guarantees, and characterize the
prediction performance on an out-of-sample datapoint in terms of this dominant
optimizer.
