  Dimension reduction (DR) aims to learn low-dimensional representations of
high-dimensional data with the preservation of essential information. In the
context of manifold learning, we define that the representation after
information-lossless DR preserves the topological and geometric properties of
data manifolds formally, and propose a novel two-stage DR method, called
invertible manifold learning (inv-ML) to bridge the gap between theoretical
information-lossless and practical DR. The first stage includes a homeomorphic
sparse coordinate transformation to learn low-dimensional representations
without destroying topology and a local isometry constraint to preserve local
geometry. In the second stage, a linear compression is implemented for the
trade-off between the target dimension and the incurred information loss in
excessive DR scenarios. Experiments are conducted on seven datasets with a
neural network implementation of inv-ML, called i-ML-Enc. Empirically, i-ML-Enc
achieves invertible DR in comparison with typical existing methods as well as
reveals the characteristics of the learned manifolds. Through latent space
interpolation on real-world datasets, we find that the reliability of tangent
space approximated by the local neighborhood is the key to the success of
manifold-based DR algorithms.
