  This paper proposes a theoretical framework on the mechanism of autoencoders.
To the encoder part, under the main use of dimensionality reduction, we
investigate its two fundamental properties: bijective maps and data
disentangling. The general construction methods of an encoder that satisfies
either or both of the above two properties are given. The generalization
mechanism of autoencoders is modeled. Based on the theoretical framework above,
we explain some experimental results of variational autoencoders, denoising
autoencoders, and linear-unit autoencoders, with emphasis on the interpretation
of the lower-dimensional representation of data via encoders; and the mechanism
of image restoration through autoencoders is natural to be understood by those
explanations. Compared to PCA and decision trees, the advantages of
(generalized) autoencoders on dimensionality reduction and classification are
demonstrated, respectively. Convolutional neural networks and randomly weighted
neural networks are also interpreted by this framework.
