  This paper presents the geometric aspect of the autoencoder framework, which,
despite its importance, has been relatively less recognized. Given a set of
high-dimensional data points that approximately lie on some lower-dimensional
manifold, an autoencoder learns the \textit{manifold} and its
\textit{coordinate chart}, simultaneously. This geometric perspective naturally
raises inquiries like "Does a finite set of data points correspond to a single
manifold?" or "Is there only one coordinate chart that can represent the
manifold?". The responses to these questions are negative, implying that there
are multiple solution autoencoders given a dataset. Consequently, they
sometimes produce incorrect manifolds with severely distorted latent space
representations. In this paper, we introduce recent geometric approaches that
address these issues.
