  Variance components estimation and mixed model analysis are central themes in
statistics with applications in numerous scientific disciplines. Despite the
best efforts of generations of statisticians and numerical analysts, maximum
likelihood estimation and restricted maximum likelihood estimation of variance
component models remain numerically challenging. Building on the
minorization-maximization (MM) principle, this paper presents a novel iterative
algorithm for variance components estimation. MM algorithm is trivial to
implement and competitive on large data problems. The algorithm readily extends
to more complicated problems such as linear mixed models, multivariate response
models possibly with missing data, maximum a posteriori estimation, penalized
estimation, and generalized estimating equations (GEE). We establish the global
convergence of the MM algorithm to a KKT point and demonstrate, both
numerically and theoretically, that it converges faster than the classical EM
algorithm when the number of variance components is greater than two and all
covariance matrices are positive definite.
