  Gaussian process latent variable models (GPLVMs) are a versatile family of
unsupervised learning models commonly used for dimensionality reduction.
However, common challenges in modeling data with GPLVMs include inadequate
kernel flexibility and improper selection of the projection noise, leading to a
type of model collapse characterized by vague latent representations that do
not reflect the underlying data structure. This paper addresses these issues
by, first, theoretically examining the impact of projection variance on model
collapse through the lens of a linear GPLVM. Second, we tackle model collapse
due to inadequate kernel flexibility by integrating the spectral mixture (SM)
kernel and a differentiable random Fourier feature (RFF) kernel approximation,
which ensures computational scalability and efficiency through off-the-shelf
automatic differentiation tools for learning the kernel hyperparameters,
projection variance, and latent representations within the variational
inference framework. The proposed GPLVM, named advisedRFLVM, is evaluated
across diverse datasets and consistently outperforms various salient competing
models, including state-of-the-art variational autoencoders (VAEs) and other
GPLVM variants, in terms of informative latent representations and missing data
imputation.
