  Topic models provide a useful tool to organize and understand the structure
of large corpora of text documents, in particular, to discover hidden thematic
structure. Clustering documents from big unstructured corpora into topics is an
important task in various areas, such as image analysis, e-commerce, social
networks, population genetics. A common approach to topic modeling is to
associate each topic with a probability distribution on the dictionary of words
and to consider each document as a mixture of topics. Since the number of
topics is typically substantially smaller than the size of the corpus and of
the dictionary, the methods of topic modeling can lead to a dramatic dimension
reduction. In this paper, we study the problem of estimating topics
distribution for each document in the given corpus, that is, we focus on the
clustering aspect of the problem. We introduce an algorithm that we call
Successive Projection Overlapping Clustering (SPOC) inspired by the Successive
Projection Algorithm for separable matrix factorization. This algorithm is
simple to implement and computationally fast. We establish theoretical
guarantees on the performance of the SPOC algorithm, in particular, near
matching minimax upper and lower bounds on its estimation risk. We also propose
a new method that estimates the number of topics. We complement our theoretical
results with a numerical study on synthetic and semi-synthetic data to analyze
the performance of this new algorithm in practice. One of the conclusions is
that the error of the algorithm grows at most logarithmically with the size of
the dictionary, in contrast to what one observes for Latent Dirichlet
Allocation.
