  Models with random effects, such as generalised linear mixed models (GLMMs),
are often used for analysing clustered data. Parameter inference with these
models is difficult because of the presence of cluster-specific random effects,
which must be integrated out when evaluating the likelihood function. Here, we
propose a sequential variational Bayes algorithm, called Recursive Variational
Gaussian Approximation for Latent variable models (R-VGAL), for estimating
parameters in GLMMs. The R-VGAL algorithm operates on the data sequentially,
requires only a single pass through the data, and can provide parameter updates
as new data are collected without the need of re-processing the previous data.
At each update, the R-VGAL algorithm requires the gradient and Hessian of a
"partial" log-likelihood function evaluated at the new observation, which are
generally not available in closed form for GLMMs. To circumvent this issue, we
propose using an importance-sampling-based approach for estimating the gradient
and Hessian via Fisher's and Louis' identities. We find that R-VGAL can be
unstable when traversing the first few data points, but that this issue can be
mitigated by using a variant of variational tempering in the initial steps of
the algorithm. Through illustrations on both simulated and real datasets, we
show that R-VGAL provides good approximations to the exact posterior
distributions, that it can be made robust through tempering, and that it is
computationally efficient.
