  To solve the big topic modeling problem, we need to reduce both time and
space complexities of batch latent Dirichlet allocation (LDA) algorithms.
Although parallel LDA algorithms on the multi-processor architecture have low
time and space complexities, their communication costs among processors often
scale linearly with the vocabulary size and the number of topics, leading to a
serious scalability problem. To reduce the communication complexity among
processors for a better scalability, we propose a novel communication-efficient
parallel topic modeling architecture based on power law, which consumes orders
of magnitude less communication time when the number of topics is large. We
combine the proposed communication-efficient parallel architecture with the
online belief propagation (OBP) algorithm referred to as POBP for big topic
modeling tasks. Extensive empirical results confirm that POBP has the following
advantages to solve the big topic modeling problem: 1) high accuracy, 2)
communication-efficient, 3) fast speed, and 4) constant memory usage when
compared with recent state-of-the-art parallel LDA algorithms on the
multi-processor architecture.
