  Dimensionality reduction (DR) algorithms compress high-dimensional data into
a lower dimensional representation while preserving important features of the
data. DR is a critical step in many analysis pipelines as it enables
visualisation, noise reduction and efficient downstream processing of the data.
In this work, we introduce the ProbDR variational framework, which interprets a
wide range of classical DR algorithms as probabilistic inference algorithms in
this framework. ProbDR encompasses PCA, CMDS, LLE, LE, MVU, diffusion maps,
kPCA, Isomap, (t-)SNE, and UMAP. In our framework, a low-dimensional latent
variable is used to construct a covariance, precision, or a graph Laplacian
matrix, which can be used as part of a generative model for the data. Inference
is done by optimizing an evidence lower bound. We demonstrate the internal
consistency of our framework and show that it enables the use of probabilistic
programming languages (PPLs) for DR. Additionally, we illustrate that the
framework facilitates reasoning about unseen data and argue that our generative
models approximate Gaussian processes (GPs) on manifolds. By providing a
unified view of DR, our framework facilitates communication, reasoning about
uncertainties, model composition, and extensions, particularly when domain
knowledge is present.
