  We introduce the Locally Linear Latent Variable Model (LL-LVM), a
probabilistic model for non-linear manifold discovery that describes a joint
distribution over observations, their manifold coordinates and locally linear
maps conditioned on a set of neighbourhood relationships. The model allows
straightforward variational optimisation of the posterior distribution on
coordinates and locally linear maps from the latent space to the observation
space given the data. Thus, the LL-LVM encapsulates the local-geometry
preserving intuitions that underlie non-probabilistic methods such as locally
linear embedding (LLE). Its probabilistic semantics make it easy to evaluate
the quality of hypothesised neighbourhood relationships, select the intrinsic
dimensionality of the manifold, construct out-of-sample extensions and to
combine the manifold model with additional probabilistic models that capture
the structure of coordinates within the manifold.
