  Most multilayer least squares (LS)-based neural networks are structured with
two separate stages: unsupervised feature encoding and supervised pattern
classification. Once the unsupervised learning is finished, the latent encoding
would be fixed without supervised fine-tuning. However, in complex tasks such
as handling the ImageNet dataset, there are often many more clues that can be
directly encoded, while the unsupervised learning, by definition cannot know
exactly what is useful for a certain task. This serves as the motivation to
retrain the latent space representations to learn some clues that unsupervised
learning has not yet learned. In particular, the error matrix from the output
layer is pulled back to each hidden layer, and the parameters of the hidden
layer are recalculated with Moore-Penrose (MP) inverse for more generalized
representations. In this paper, a recomputation-based multilayer network using
MP inverse (RML-MP) is developed. A sparse RML-MP (SRML-MP) model to boost the
performance of RML-MP is then proposed. The experimental results with varying
training samples (from 3 K to 1.8 M) show that the proposed models provide
better generalization performance than most representation learning algorithms.
