  This paper introduces Least Volume-a simple yet effective regularization
inspired by geometric intuition-that can reduce the necessary number of latent
dimensions needed by an autoencoder without requiring any prior knowledge of
the intrinsic dimensionality of the dataset. We show that the Lipschitz
continuity of the decoder is the key to making it work, provide a proof that
PCA is just a linear special case of it, and reveal that it has a similar
PCA-like importance ordering effect when applied to nonlinear models. We
demonstrate the intuition behind the regularization on some pedagogical toy
problems, and its effectiveness on several benchmark problems, including MNIST,
CIFAR-10 and CelebA.
