  Mini-batch algorithms have become increasingly popular due to the requirement
for solving optimization problems, based on large-scale data sets. Using an
existing online expectation-{}-maximization (EM) algorithm framework, we
demonstrate how mini-batch (MB) algorithms may be constructed, and propose a
scheme for the stochastic stabilization of the constructed mini-batch
algorithms. Theoretical results regarding the convergence of the mini-batch EM
algorithms are presented. We then demonstrate how the mini-batch framework may
be applied to conduct maximum likelihood (ML) estimation of mixtures of
exponential family distributions, with emphasis on ML estimation for mixtures
of normal distributions. Via a simulation study, we demonstrate that the
mini-batch algorithm for mixtures of normal distributions can outperform the
standard EM algorithm. Further evidence of the performance of the mini-batch
framework is provided via an application to the famous MNIST data set.
