  Choosing the number of mixture components remains an elusive challenge. Model
selection criteria can be either overly liberal or conservative and return
poorly-separated components of limited practical use. We formalize non-local
priors (NLPs) for mixtures and show how they lead to well-separated components
with non-negligible weight, interpretable as distinct subpopulations. We also
propose an estimator for posterior model probabilities under local and
non-local priors, showing that Bayes factors are ratios of posterior to prior
empty-cluster probabilities. The estimator is widely applicable and helps set
thresholds to drop unoccupied components in overfitted mixtures. We suggest
default prior parameters based on multi-modality for Normal/T mixtures and
minimal informativeness for categorical outcomes. We characterise theoretically
the NLP-induced sparsity, derive tractable expressions and algorithms. We fully
develop Normal, Binomial and product Binomial mixtures but the theory,
computation and principles hold more generally. We observed a serious lack of
sensitivity of the Bayesian information criterion (BIC), insufficient parsimony
of the AIC and a local prior, and a mixed behavior of the singular BIC. We also
considered overfitted mixtures, their performance was competitive but depended
on tuning parameters. Under our default prior elicitation NLPs offered a good
compromise between sparsity and power to detect meaningfully-separated
components.
