  Expectation maximization (EM) is a technique for estimating
maximum-likelihood parameters of a latent variable model given observed data by
alternating between taking expectations of sufficient statistics, and
maximizing the expected log likelihood. For situations where sufficient
statistics are intractable, stochastic approximation EM (SAEM) is often used,
which uses Monte Carlo techniques to approximate the expected log likelihood.
Two common implementations of SAEM, Batch EM (BEM) and online EM (OEM), are
parameterized by a "learning rate", and their efficiency depend strongly on
this parameter. We propose an extension to the OEM algorithm, termed
Introspective Online Expectation Maximization (IOEM), which removes the need
for specifying this parameter by adapting the learning rate according to trends
in the parameter updates. We show that our algorithm matches the efficiency of
the optimal BEM and OEM algorithms in multiple models, and that the efficiency
of IOEM can exceed that of BEM/OEM methods with optimal learning rates when the
model has many parameters. A Python implementation is available at
https://github.com/luntergroup/IOEM.git.
