  There has been great interest recently in applying nonparametric kernel
mixtures in a hierarchical manner to model multiple related data samples
jointly. In such settings several data features are commonly present: (i) the
related samples often share some, if not all, of the mixture components but
with differing weights, (ii) only some, not all, of the mixture components vary
across the samples, and (iii) often the shared mixture components across
samples are not aligned perfectly in terms of their location and spread, but
rather display small misalignments either due to systematic cross-sample
difference or more often due to uncontrolled, extraneous causes. Properly
incorporating these features in mixture modeling will enhance the efficiency of
inference, whereas ignoring them not only reduces efficiency but can jeopardize
the validity of the inference due to issues such as confounding. We introduce
two techniques for incorporating these features in modeling related data
samples using kernel mixtures. The first technique, called $\psi$-stick
breaking, is a joint generative process for the mixing weights through the
breaking of both a stick shared by all the samples for the components that do
not vary in size across samples and an idiosyncratic stick for each sample for
those components that do vary in size. The second technique is to imbue random
perturbation into the kernels, thereby accounting for cross-sample
misalignment. These techniques can be used either separately or together in
both parametric and nonparametric kernel mixtures. We derive efficient Bayesian
inference recipes based on MCMC sampling for models featuring these techniques,
and illustrate their work through both simulated data and a real flow cytometry
data set in prediction/estimation, cross-sample calibration, and testing
multi-sample differences.
