  Active learning seeks to reduce the amount of data required to fit the
parameters of a model, thus forming an important class of techniques in modern
machine learning. However, past work on active learning has largely overlooked
latent variable models, which play a vital role in neuroscience, psychology,
and a variety of other engineering and scientific disciplines. Here we address
this gap by proposing a novel framework for maximum-mutual-information input
selection for discrete latent variable regression models. We first apply our
method to a class of models known as "mixtures of linear regressions" (MLR).
While it is well known that active learning confers no advantage for
linear-Gaussian regression models, we use Fisher information to show
analytically that active learning can nevertheless achieve large gains for
mixtures of such models, and we validate this improvement using both
simulations and real-world data. We then consider a powerful class of
temporally structured latent variable models given by a Hidden Markov Model
(HMM) with generalized linear model (GLM) observations, which has recently been
used to identify discrete states from animal decision-making data. We show that
our method substantially reduces the amount of data needed to fit GLM-HMM, and
outperforms a variety of approximate methods based on variational and amortized
inference. Infomax learning for latent variable models thus offers a powerful
for characterizing temporally structured latent states, with a wide variety of
applications in neuroscience and beyond.
