  Although variational autoencoders (VAE) are successfully used to obtain
meaningful low-dimensional representations for high-dimensional data, the
characterization of critical points of the loss function for general
observation models is not fully understood. We introduce a theoretical
framework that is based on a connection between $\beta$-VAE and generalized
linear models (GLM). The equality between the activation function of a
$\beta$-VAE and the inverse of the link function of a GLM enables us to provide
a systematic generalization of the loss analysis for $\beta$-VAE based on the
assumption that the observation model distribution belongs to an exponential
dispersion family (EDF). As a result, we can initialize $\beta$-VAE nets by
maximum likelihood estimates (MLE) that enhance the training performance on
both synthetic and real world data sets. As a further consequence, we
analytically describe the auto-pruning property inherent in the $\beta$-VAE
objective and reason for posterior collapse.
