  Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for
hidden semantic discovery of text data and serves as a fundamental tool for
text analysis in various applications. However, the LDA model as well as the
training process of LDA may expose the text information in the training data,
thus bringing significant privacy concerns. To address the privacy issue in
LDA, we systematically investigate the privacy protection of the main-stream
LDA training algorithm based on Collapsed Gibbs Sampling (CGS) and propose
several differentially private LDA algorithms for typical training scenarios.
In particular, we present the first theoretical analysis on the inherent
differential privacy guarantee of CGS based LDA training and further propose a
centralized privacy-preserving algorithm (HDP-LDA) that can prevent data
inference from the intermediate statistics in the CGS training. Also, we
propose a locally private LDA training algorithm (LP-LDA) on crowdsourced data
to provide local differential privacy for individual data contributors.
Furthermore, we extend LP-LDA to an online version as OLP-LDA to achieve LDA
training on locally private mini-batches in a streaming setting. Extensive
analysis and experiment results validate both the effectiveness and efficiency
of our proposed privacy-preserving LDA training algorithms.
