  Autoencoders and generative models produce some of the most spectacular deep
learning results to date. However, understanding and controlling the latent
space of these models presents a considerable challenge. Drawing inspiration
from principal component analysis and autoencoder, we propose the Principal
Component Analysis Autoencoder (PCAAE). This is a novel autoencoder whose
latent space verifies two properties. Firstly, the dimensions are organised in
decreasing importance with respect to the data at hand. Secondly, the
components of the latent space are statistically independent. We achieve this
by progressively increasing the latent space during training, and with a
covariance loss applied to the latent codes. The resulting autoencoder produces
a latent space which separates the intrinsic attributes of the data into
different components of the latent space, in a completely unsupervised manner.
We also describe an extension of our approach to the case of powerful,
pre-trained GANs. We show results on both synthetic examples of shapes and on a
state-of-the-art GAN. For example, we are able to separate the color shade
scale of hair and skin, pose of faces and the gender in the CelebA, without
accessing any labels. We compare the PCAAE with other state-of-the-art
approaches, in particular with respect to the ability to disentangle attributes
in the latent space. We hope that this approach will contribute to better
understanding of the intrinsic latent spaces of powerful deep generative
models.
