  Semiparametric regression offers a flexible framework for modeling non-linear
relationships between a response and covariates. A prime example are
generalized additive models where splines (say) are used to approximate
non-linear functional components in conjunction with a quadratic penalty to
control for overfitting. Estimation and inference are then generally performed
based on the penalized likelihood, or under a mixed model framework. The
penalized likelihood framework is fast but potentially unstable, and choosing
the smoothing parameters needs to be done externally using cross-validation,
for instance. The mixed model framework tends to be more stable and offers a
natural way for choosing the smoothing parameters, but for non-normal responses
involves an intractable integral.
  In this article, we introduce a new framework for semiparametric regression
based on variational approximations. The approach possesses the stability and
natural inference tools of the mixed model framework, while achieving
computation times comparable to using penalized likelihood. Focusing on
generalized additive models, we derive fully tractable variational likelihoods
for some common response types. We present several features of the variational
approximation framework for inference, including a variational information
matrix for inference on parametric components, and a closed-form update for
estimating the smoothing parameter. We demonstrate the consistency of the
variational approximation estimates, and an asymptotic normality result for the
parametric component of the model. Simulation studies show the variational
approximation framework performs similarly to and sometimes better than
currently available software for fitting generalized additive models.
