  To address three important issues involved in latent variable models (LVMs),
including capturing infrequent patterns, achieving small-sized but expressive
models and alleviating overfitting, several studies have been devoted to
"diversifying" LVMs, which aim at encouraging the components in LVMs to be
diverse. Most existing studies fall into a frequentist-style regularization
framework, where the components are learned via point estimation. In this
paper, we investigate how to "diversify" LVMs in the paradigm of Bayesian
learning. We propose two approaches that have complementary advantages. One is
to define a diversity-promoting mutual angular prior which assigns larger
density to components with larger mutual angles and use this prior to affect
the posterior via Bayes' rule. We develop two efficient approximate posterior
inference algorithms based on variational inference and MCMC sampling. The
other approach is to impose diversity-promoting regularization directly over
the post-data distribution of components. We also extend our approach to
"diversify" Bayesian nonparametric models where the number of components is
infinite. A sampling algorithm based on slice sampling and Hamiltonian Monte
Carlo is developed. We apply these methods to "diversify" Bayesian mixture of
experts model and infinite latent feature model. Experiments on various
datasets demonstrate the effectiveness and efficiency of our methods.
