  Topic modeling is a very powerful technique in data analysis and data mining
but it is generally slow. Many parallelization approaches have been proposed to
speed up the learning process. However, they are usually not very efficient
because of the many kinds of overhead, especially the load-balancing problem.
We address this problem by proposing three partitioning algorithms, which
either run more quickly or achieve better load balance than current
partitioning algorithms. These algorithms can easily be extended to improve
parallelization efficiency on other topic models similar to LDA, e.g., Bag of
Timestamps, which is an extension of LDA with time information. We evaluate
these algorithms on two popular datasets, NIPS and NYTimes. We also build a
dataset containing over 1,000,000 scientific publications in the computer
science domain from 1951 to 2010 to experiment with Bag of Timestamps
parallelization, which we design to demonstrate the proposed algorithms'
extensibility. The results strongly confirm the advantages of these algorithms.
