  Given an observational study with $n$ independent but heterogeneous units,
our goal is to learn the counterfactual distribution for each unit using only
one $p$-dimensional sample per unit containing covariates, interventions, and
outcomes. Specifically, we allow for unobserved confounding that introduces
statistical biases between interventions and outcomes as well as exacerbates
the heterogeneity across units. Modeling the conditional distribution of the
outcomes as an exponential family, we reduce learning the unit-level
counterfactual distributions to learning $n$ exponential family distributions
with heterogeneous parameters and only one sample per distribution. We
introduce a convex objective that pools all $n$ samples to jointly learn all
$n$ parameter vectors, and provide a unit-wise mean squared error bound that
scales linearly with the metric entropy of the parameter space. For example,
when the parameters are $s$-sparse linear combination of $k$ known vectors, the
error is $O(s\log k/p)$. En route, we derive sufficient conditions for
compactly supported distributions to satisfy the logarithmic Sobolev
inequality. As an application of the framework, our results enable consistent
imputation of sparsely missing covariates.
