  Within a Bayesian framework, a comprehensive investigation of mixtures of
finite mixtures (MFMs), i.e., finite mixtures with a prior on the number of
components, is performed. This model class has applications in model-based
clustering as well as for semi-parametric density estimation and requires
suitable prior specifications and inference methods to exploit its full
potential. We contribute by considering a generalized class of MFMs where the
hyperparameter $\gamma_K$ of a symmetric Dirichlet prior on the weight
distribution depends on the number of components. We show that this model class
may be regarded as a Bayesian non-parametric mixture outside the class of
Gibbs-type priors. We emphasize the distinction between the number of
components $K$ of a mixture and the number of clusters $K_+$, i.e., the number
of filled components given the data. In the MFM model, $K_+$ is a random
variable and its prior depends on the prior on $K$ and on the hyperparameter
$\gamma_K$. We employ a flexible prior distribution for the number of
components $K$ and derive the corresponding prior on the number of clusters
$K_+$ for generalized MFMs. For posterior inference, we propose the novel
telescoping sampler which allows Bayesian inference for mixtures with arbitrary
component distributions without resorting to reversible jump Markov chain Monte
Carlo (MCMC) methods. The telescoping sampler explicitly samples the number of
components, but otherwise requires only the usual MCMC steps of a finite
mixture model. The ease of its application using different component
distributions is demonstrated on several data sets.
