  Variational inference is a popular method for estimating model parameters and
conditional distributions in hierarchical and mixed models, which arise
frequently in many settings in the health, social, and biological sciences.
Variational inference in a frequentist context works by approximating
intractable conditional distributions with a tractable family and optimizing
the resulting lower bound on the log-likelihood. The variational objective
function is typically less computationally intensive to optimize than the true
likelihood, enabling scientists to fit rich models even with extremely large
datasets. Despite widespread use, little is known about the general theoretical
properties of estimators arising from variational approximations to the
log-likelihood, which hinders their use in inferential statistics. In this
paper we connect such estimators to profile M-estimation, which enables us to
provide regularity conditions for consistency and asymptotic normality of
variational estimators. Our theory also motivates three methodological
improvements to variational inference: estimation of the asymptotic
model-robust covariance matrix, a one-step correction that improves estimator
efficiency, and an empirical assessment of consistency. We evaluate the
proposed results using simulation studies and data on marijuana use from the
National Longitudinal Study of Youth.
