  We propose a novel topic discovery algorithm for unlabeled images based on
the bag-of-words (BoW) framework. We first extract a dictionary of visual words
and subsequently for each image compute a visual word occurrence histogram. We
view these histograms as rows of a large matrix from which we extract sparse
principal components (PCs). Each PC identifies a sparse combination of visual
words which co-occur frequently in some images but seldom appear in others.
Each sparse PC corresponds to a topic, and images whose interference with the
PC is high belong to that topic, revealing the common parts possessed by the
images. We propose to solve the associated sparse PCA problems using an
Alternating Maximization (AM) method, which we modify for purpose of
efficiently extracting multiple PCs in a deflation scheme. Our approach attacks
the maximization problem in sparse PCA directly and is scalable to
high-dimensional data. Experiments on automatic topic discovery and category
prediction demonstrate encouraging performance of our approach.
