  Latent variable models (LVMs) represent observed variables by parameterized
functions of latent variables. Prominent examples of LVMs for unsupervised
learning are probabilistic PCA or probabilistic SC which both assume a weighted
linear summation of the latents to determine the mean of a Gaussian
distribution for the observables. In many cases, however, observables do not
follow a Gaussian distribution. For unsupervised learning, LVMs which assume
specific non-Gaussian observables have therefore been considered. Already for
specific choices of distributions, parameter optimization is challenging and
only a few previous contributions considered LVMs with more generally defined
observable distributions. Here, we consider LVMs that are defined for a range
of different distributions, i.e., observables can follow any (regular)
distribution of the exponential family. The novel class of LVMs presented is
defined for binary latents, and it uses maximization in place of summation to
link the latents to observables. To derive an optimization procedure, we follow
an EM approach for maximum likelihood parameter estimation. We show that a set
of very concise parameter update equations can be derived which feature the
same functional form for all exponential family distributions. The derived
generic optimization can consequently be applied to different types of metric
data as well as to different types of discrete data. Also, the derived
optimization equations can be combined with a recently suggested variational
acceleration which is likewise generically applicable to the LVMs considered
here. So, the combination maintains generic and direct applicability of the
derived optimization procedure, but, crucially, enables efficient scalability.
We numerically verify our analytical results and discuss some potential
applications such as learning of variance structure, noise type estimation and
denoising.
