  Restricted maximum likelihood (REML) estimation is a widely accepted and
frequently used method for fitting linear mixed models, with its principal
advantage being that it produces less biased estimates of the variance
components. However, the concept of REML does not immediately generalize to the
setting of non-normally distributed responses, and it is not always clear the
extent to which, either asymptotically or in finite samples, such
generalizations reduce the bias of variance component estimates compared to
standard unrestricted maximum likelihood estimation. In this article, we review
various attempts that have been made over the past four decades to extend REML
estimation in generalized linear mixed models. We establish four major classes
of approaches, namely approximate linearization, integrated likelihood,
modified profile likelihoods, and direct bias correction of the score function,
and show that while these four classes may have differing motivations and
derivations, they often arrive at a similar if not the same REML estimate. We
compare the finite sample performance of these four classes through a numerical
study involving binary and count data, with results demonstrating that they
perform similarly well in reducing the finite sample bias of variance
components.
