  In analyzing complex datasets, it is often of interest to infer lower
dimensional structure underlying the higher dimensional observations. As a
flexible class of nonlinear structures, it is common to focus on Riemannian
manifolds. Most existing manifold learning algorithms replace the original data
with lower dimensional coordinates without providing an estimate of the
manifold in the observation space or using the manifold to denoise the original
data. This article proposes a new methodology for addressing these problems,
allowing interpolation of the estimated manifold between fitted data points.
The proposed approach is motivated by novel theoretical properties of local
covariance matrices constructed from noisy samples on a manifold. Our results
enable us to turn a global manifold reconstruction problem into a local
regression problem, allowing application of Gaussian processes for
probabilistic manifold reconstruction. In addition to theory justifying the
algorithm, we provide simulated and real data examples to illustrate the
performance.
