  The marginal likelihood or evidence in Bayesian statistics contains an
intrinsic penalty for larger model sizes and is a fundamental quantity in
Bayesian model comparison. Over the past two decades, there has been steadily
increasing activity to understand the nature of this penalty in singular
statistical models, building on pioneering work by Sumio Watanabe. Unlike
regular models where the Bayesian information criterion (BIC) encapsulates a
first-order expansion of the logarithm of the marginal likelihood, parameter
counting gets trickier in singular models where a quantity called the real log
canonical threshold (RLCT) summarizes the effective model dimensionality. In
this article, we offer a probabilistic treatment to recover non-asymptotic
versions of established evidence bounds as well as prove a new result based on
the Gibbs variational inequality. In particular, we show that mean-field
variational inference correctly recovers the RLCT for any singular model in its
canonical or normal form. We additionally exhibit sharpness of our bound by
analyzing the dynamics of a general purpose coordinate ascent algorithm (CAVI)
popularly employed in variational inference.
