  Learning invariant representations is an important problem in machine
learning and pattern recognition. In this paper, we present a novel framework
of transformation-invariant feature learning by incorporating linear
transformations into the feature learning algorithms. For example, we present
the transformation-invariant restricted Boltzmann machine that compactly
represents data by its weights and their transformations, which achieves
invariance of the feature representation via probabilistic max pooling. In
addition, we show that our transformation-invariant feature learning framework
can also be extended to other unsupervised learning methods, such as
autoencoders or sparse coding. We evaluate our method on several image
classification benchmark datasets, such as MNIST variations, CIFAR-10, and
STL-10, and show competitive or superior classification performance when
compared to the state-of-the-art. Furthermore, our method achieves
state-of-the-art performance on phone classification tasks with the TIMIT
dataset, which demonstrates wide applicability of our proposed algorithms to
other domains.
