  We consider the problem of constructing distribution-free prediction sets for
data from two-layer hierarchical distributions. For iid data, prediction sets
can be constructed using the method of conformal prediction. The validity of
conformal prediction hinges on the exchangeability of the data, which does not
hold when groups of observations come from distinct distributions, such as
multiple observations on each patient in a medical database. We extend
conformal methods to this hierarchical setting. We develop CDF pooling, single
subsampling, and repeated subsampling approaches to construct prediction sets
in unsupervised and supervised settings. We compare these approaches in terms
of coverage and average set size. If asymptotic coverage is acceptable, we
recommend CDF pooling for its balance between empirical coverage and average
set size. If we desire coverage guarantees, then we recommend the repeated
subsampling approach.
